在Perl中,我如何等待线程并行结束？

Question

我有一个Perl脚本,可以启动2个线程,每个处理器一个.我需要它等待一个线程结束,如果一个线程结束,则生成一个新线程.似乎join方法阻塞了程序的其余部分,因此第二个线程不能结束,直到第一个线程完成的所有操作都失败了.

我试过这个is_joinable方法,但似乎也没办法.

这是我的一些代码:

use threads;
use threads::shared;

@file_list = @ARGV;      #Our file list
$nofiles = $#file_list + 1; #Real number of files 
$currfile = 1;     #Current number of file to process

my %MSG : shared;              #shared hash

$thr0 = threads->new(\&process, shift(@file_list));
$currfile++;
$thr1 = threads->new(\&process, shift(@file_list));
$currfile++;

while(1){
 if ($thr0->is_joinable()) {
  $thr0->join;
        #check if there are files left to process
  if($currfile <= $nofiles){ 
   $thr0 = threads->new(\&process, shift(@file_list));
   $currfile++;
  }
 }

 if ($thr1->is_joinable()) {
  $thr1->join;
        #check if there are files left to process
  if($currfile <= $nofiles){
   $thr1 = threads->new(\&process, shift(@file_list));
   $currfile++;
  }
 }
}

sub process{
       print "Opening $currfile of $nofiles\n";
       #do some stuff
       if(some condition){
               lock(%MSG);
               #write stuff to hash
       }
       print "Closing $currfile of $nofiles\n";
}

这个输出是:

Opening 1 of 4
Opening 2 of 4
Closing 1 of 4
Opening 3 of 4
Closing 3 of 4
Opening 4 of 4
Closing 2 of 4
Closing 4 of 4

Answer 1

首先,对代码本身进行一些评论.你需要确保你有:

use strict;
use warnings;

在每个脚本的开头.第二:

@file_list = @ARGV;      #Our file list
$nofiles = $#file_list + 1; #Real number of files 

因为标量上下文中的数组计算数组中元素的数量,所以是不必要的.那是:

$nofiles = @ARGV;

@ARGV无论值如何,都会正确地为您提供文件数$[.

最后,通过在启动线程之前对文件列表进行分区,可以使脚本更简单:

use strict; use warnings;

use threads;
use threads::shared;

my @threads = (
    threads->new(\&process, @ARGV[0 .. @ARGV/2]),
    threads->new(\&process, @ARGV[@ARGV/2 + 1 .. @ARGV - 1]),
);

$_->join for @threads;

sub process {
    my @files = @_;
    warn "called with @files\n";
    for my $file ( @files ) {
        warn "opening '$file'\n";
        sleep rand 3;
        warn "closing '$file'\n";
    }
}

输出:

C:\Temp> thr 1 2 3 4 5
called with 1 2 3
opening '1'
called with 4 5
opening '4'
closing '4'
opening '5'
closing '1'
opening '2'
closing '5'
closing '2'
opening '3'
closing '3'

或者,您可以让线程在完成后继续执行下一个任务:

use strict; use warnings;

use threads;
use threads::shared;

my $current :shared;
$current = 0;

my @threads = map { threads->new(\&process, $_) } 1 .. 2;
$_->join for @threads;

sub process {
    my ($thr) = @_;
    warn "thread $thr stared\n";

    while ( 1 ) {
        my $file;
        {
            lock $current;
            return unless $current < @ARGV;
            $file = $ARGV[$current];
            ++ $current;
        }
        warn "$thr: opening '$file'\n";
        sleep rand 5;
        warn "$thr: closing '$file'\n";
    }
}

输出:

C:\Temp> thr 1 2 3 4 5
thread 1 stared
1: opening '1'
1: closing '1'
1: opening '2'
thread 2 stared
2: opening '3'
2: closing '3'
2: opening '4'
1: closing '2'
1: opening '5'
1: closing '5'
2: closing '4'

Answer 2

我认为您需要将从列表中提取下一个文件的代码移动到线程本身.

因此,每个线程不会只处理一个文件,而是继续处理,直到列表为空.

这样,您还可以节省始终创建新线程的开销.

然后你的主线程将加入它们.

当然,这需要在列表上进行同步(以便它们不会提取相同的数据).或者,您可以将列表拆分为两个(每个线程一个),但这可能会导致不幸的分发.

(PS:没有Perl上帝,只是一个不起眼的僧侣)