Python Multiprocessing.Process 如何重用进程？

Question

我正在使用 python 多处理模块并行运行一些长时间运行的任务。我正在使用 start() 方法来运行作业，但是一旦作业返回，我想再次运行它们。

\n\n

是否可以重用我创建的流程？或者每次我想运行作业时都必须创建一个新的 Process 对象？

\n\n

pyhton 文档中有这一部分建议我不能多次使用 start() 方法，但也许有人知道重用实例的另一种方法：

\n\n

开始（）

\n\n

启动 process\xe2\x80\x99s 活动。

\n\n

每个进程对象最多只能调用一次。\n它安排在单独的进程中调用对象\xe2\x80\x99s run() 方法。

\n\n

这是我的 Process 类版本：

\n\n

class Process(multiprocessing.Process):\n    def __init__(self, result_queue, MCMCinstance):\n        assert isinstance(MCMCinstance, MCMC)\n        multiprocessing.Process.__init__(self)\n        self.result_queue = result_queue\n        self.mcmc = MCMCinstance\n        self.interface = C_interface(self.mcmc)\n        self.burn_in = False\n\n    def run(self):\n        if self.burn_in: interface.burn_in()\n        self.interface.sample(self.mcmc.options.runs)\n        self.interface.update(self.mcmc)\n        self.result_queue.put(self.mcmc)\n

\n\n

然后我实例化进程并使用 start() 方法运行它们：

\n\n

# setup the jobs and run\nresult_queue = multiprocessing.Queue()\n\nmcmc1 = MCMC(options, donors, clusters)\nmcmc2 = MCMC(options, donors, clusters)\nmcmc3 = MCMC(options, donors, clusters)\nmcmc4 = MCMC(options, donors, clusters)\n\np1 = Process(result_queue, mcmc1)\np2 = Process(result_queue, mcmc2)\np3 = Process(result_queue, mcmc3)\np4 = Process(result_queue, mcmc4)\n\njobs = [p1, p2, p3, p4]\n\nfor job in jobs:\n    job.start()\n\nresults = [result_queue.get() for job in jobs]\n

\n

Answer 1

就像文档所说，你只能调用 .start() 方法一次，我相信我每次都必须创建新的进程：

# setup the jobs and run
result_queue = multiprocessing.Queue()

mcmc1 = MCMC(options, donors, clusters)
mcmc2 = MCMC(options, donors, clusters)
mcmc3 = MCMC(options, donors, clusters)
mcmc4 = MCMC(options, donors, clusters)

p1 = Process(result_queue, mcmc1)
p2 = Process(result_queue, mcmc2)
p3 = Process(result_queue, mcmc3)
p4 = Process(result_queue, mcmc4)

jobs = [p1, p2, p3, p4]

for job in jobs:
    #job.debug_level = 1
    job.start()

results = [result_queue.get() for job in jobs]

#for res in results: res.traceplot(show=False)
p5 = Process(result_queue, results[0])
p6 = Process(result_queue, results[1])
p7 = Process(result_queue, results[2])
p8 = Process(result_queue, results[3])

jobs2 = [p5, p6, p7, p8]

for j in jobs2:
    j.start()


results2 = [result_queue.get() for job in jobs2]