我在 python 3.7 中使用多进程池映射,我在使用 docker 容器时遇到如下错误
multiprocessing.pool.MaybeEncoding Error: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7f8ff58e8110>'. Reason: 'PicklingError("Can't pickle <class 'MemoryError'>: it's not the same object as builtins.MemoryError")'
Run Code Online (Sandbox Code Playgroud)
我正在使用如下代码,
def foo(track):
track_url = track[1]
url = requests.get(track_url)
hashes = fingerprint.fingerprint_worker(
BytesIO(url.content)
)
return True
def worker:
tracks = [[1, "track_url_1"], [2, "track_url_2"]]
with multiprocessing.Pool(const.MULTIPROCESSING_POOL_SIZE) as pool:
pool.map(foo, tracks)
pool.close()
pool.terminate()
pool.join()
Run Code Online (Sandbox Code Playgroud)
MULTIPROCESSING_POOL_SIZE = 1
此错误的完整堆栈跟踪是,
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", …Run Code Online (Sandbox Code Playgroud)