如何处理 multiprocessing.Pool 中的初始化错误?

tab*_*ata 3 python python-multiprocessing

当初始化程序抛出如下错误时,脚本将不会停止。
我想在开始主进程之前中止(不要运行“do_something”)。

from multiprocessing import Pool
import contextlib

def initializer():
    raise Exception("init failed")

def do_something(args):
    # main process
    pass

pool = Pool(1, initializer=initializer)
with contextlib.closing(pool):
    try:
        pool.map_async(do_something, [1]).get(100)
    except:
        pool.terminate()
Run Code Online (Sandbox Code Playgroud)

控制台上永不停止的堆栈跟踪如下

...
Exception: init failed
Process ForkPoolWorker-18:
Traceback (most recent call last):
  File "/home/hoge/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/home/hoge/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/hoge/anaconda3/lib/python3.6/multiprocessing/pool.py", line 103, in worker
    initializer(*initargs)
  File "hoge.py", line 5, in initializer
    raise Exception("init failed")
Exception: init failed
...
Run Code Online (Sandbox Code Playgroud)

我的解决方法是通过使用如下所示的全局标志来抑制初始化程序错误并在主进程开始时返回。
但我想学更好的。

def initializer():
    try:
        raise Exception("init failed")
    except:
        global failed
        failed = True

def do_something(args):
    global failed
    if failed:
        # skip when initializer failed
        return
    # main process
Run Code Online (Sandbox Code Playgroud)

Tin*_* LI 5

在使用 PyCharm 浏览多处理的实现之后,我确信没有更好的解决方案,因为如果存在任何工作进程(无论是意外还是初始化失败),Pool 都会通过 _repopulate_pool() 启动一个线程来 _maintain_pool() 。

检查一下:Lib/multiprocessing/pool.py 第 244 行