use*_*191 7 python parallel-processing class class-method concurrent.futures
我想让我的类方法并行运行,但是它只会产生某种我无法解决的错误。我的代码是:
import concurrent.futures as futures
samples = ['asfd', 'zxcv', 'asf', 'qwer']
class test:
def __init__(self, samples):
maturedb = {}
with futures.ProcessPoolExecutor() as exe:
for samplename, dResult in exe.map(self.make_readdb, samples):
maturedb[samplename] = dResult
print(maturedb)
def make_readdb(self, samplename):
return samplename, 1
test(samples)
Run Code Online (Sandbox Code Playgroud)
如果我在Ubuntu计算机上运行此代码,则会发生如下错误:
Traceback (most recent call last):
File "/usr/lib/python3.2/multiprocessing/queues.py", line 272, in _feedsend(obj)
_pickle.PicklingError: Can't pickle <class 'method'>: attribute lookup builtins.method failed
Run Code Online (Sandbox Code Playgroud)
方法make_readdb只是简化为一个示例,但是它是实际代码中的瓶颈,我需要使其平行。请帮忙。
来自文档:
ProcessPoolExecutor 类是 Executor 子类,它使用进程池异步执行调用。ProcessPoolExecutor 使用多处理模块,这允许它绕过全局解释器锁,但也意味着只能执行和返回可挑选的对象。
我再次查看了你的代码,问题是该函数 - make_readdb- 是该类的成员test。你能重构并把这个功能拉出来吗?