Alm*_*hat 6 python concurrent.futures
我想到的用例如下:我想使用ThreadPoolExecutor启动一系列作业,然后当作业完成时,我想向队列添加一个新作业。我还想知道下一项工作何时完成并重复上述过程。当我有机会观察到预定数量的结果后,我想正确地终止一切。例如,考虑以下代码,其中run_con方法中注释掉的位显示了我想要实现的目标。
import numpy as np
import time
from concurrent import futures
MAX_WORKERS = 20
seed = 1234
np.random.seed(seed)
MSG = "Wall time: {:.2f}s"
def expensive_function(x):
time.sleep(x)
return x
def run_con(func, ts):
t0 = time.time()
workers = min(MAX_WORKERS, len(ts))
with futures.ThreadPoolExecutor(workers) as executor:
jobs = []
for t in ts:
jobs.append(executor.submit(func, t))
done = futures.as_completed(jobs)
for future in done:
print("Job complete: ", future.result())
# depending on some condition, add new job to jobs, e.g.
# jobs.append(func, np.random.random())
# update done generator
# if a threshold on total jobs is reached, close every thing down sensibly.
print(MSG.format(time.time()-t0))
ts = np.random.random(10)*5
print("Sleep times: ", ts)
run_con(expensive_function, ts)
Run Code Online (Sandbox Code Playgroud)
这可以通过并发.futures 实现吗?如果没有,还有哪些替代方案?