什么是Perl中并行处理的最佳模块？

Question

Perl中并行处理的最佳模块是什么？我从未在Perl中完成并行处理.

什么是用于数据库访问和邮件的并行进程的优秀Perl模块？

我查看了模块Parallel :: ForkManager.任何想法都赞赏.

Answer 1

正如POD所说,Parallel :: ForkManager可以限制分叉的进程数.然后你可以让孩子们做任何工作.我记得使用该模块进行下载.

Answer 2

嗯,这一切都取决于具体情况.根据您需要实现的目标,您可能会更好地使用以下方法之一:

• 并行::任务管理器
• POE
• AnyEvent

对于某些任务,还有专门的模块 - 例如LWP :: Parallel :: UserAgent,这基本上意味着 - 你必须向我们提供更多关于你想要实现什么以获得最佳答案的细节.

Answer 3

Perne(MCE)的多核心引擎已发布在CPAN上.

http://code.google.com/p/many-core-engine-perl/

https://metacpan.org/module/MCE

MCE附带了各种示例,显示了实际的用例场景,可以将像cat一样小的东西(尝试使用-n)与greped模式和字数统计聚合并行化.

barrier_sync.pl
          A barrier sync demonstration.

cat.pl    Concatenation script, similar to the cat binary.
egrep.pl  Egrep script, similar to the egrep binary.
wc.pl     Word count script, similar to the wc binary.

findnull.pl
          A parallel driven script to report lines containing
          null fields. It is many times faster than the binary
          egrep command. Try against a large file containing
          very long lines.

flow_model.pl
          Demonstrates MCE::Flow, MCE::Queue, and MCE->gather.

foreach.pl, forseq.pl, forchunk.pl
          These take the same sqrt example from Parallel::Loops
          and measures the overhead of the engine. The number
          indicates the size of @input which can be submitted
          and results displayed in 1 second.

          Parallel::Loops:     600  Forking each @input is expensive
          MCE foreach....:  34,000  Sends result after each @input
          MCE forseq.....:  70,000  Loops through sequence of numbers
          MCE forchunk...: 480,000  Chunking reduces overhead

interval.pl
          Demonstration of the interval option appearing in MCE 1.5.

matmult/matmult_base.pl, matmult_mce.pl, strassen_mce.pl
          Various matrix multiplication demonstrations benchmarking
          PDL, PDL + MCE, as well as parallelizing Strassen
          divide-and-conquer algorithm. Also included are 2 plain
          Perl examples.

scaling_pings.pl
          Perform ping test and report back failing IPs to
          standard output.

seq_demo.pl
          A demonstration of the new sequence option appearing
          in MCE 1.3. Run with seq_demo.pl | sort

tbray/wf_mce1.pl, wf_mce2.pl, wf_mce3.pl
          An implementation of wide finder utilizing MCE.
          As fast as MMAP IO when file resides in OS FS cache.
          2x ~ 3x faster when reading directly from disk.