如何有效地迭代多个生成器？

Question

我有三个不同的生成器,它们可以从网络上获取数据.因此,每次迭代可能需要一段时间才能完成.

我想混合调用生成器,并考虑roundrobin(在这里找到).问题是每次通话都会被阻止,直到完成为止.

有没有办法在不阻塞的情况下同时遍历所有生成器？

Answer 1

你可以用iter()我ThreadPool班上的方法做到这一点.

pool.iter()产生线程函数返回值,直到所有装饰+被调用函数完成执行.装饰所有异步函数,调用它们,然后循环pool.iter()以捕获它们发生的值.

例:

import time
from threadpool import ThreadPool
pool = ThreadPool(max_threads=25, catch_returns=True)

# decorate any functions you need to aggregate
# if you're pulling a function from an outside source
# you can still say 'func = pool(func)' or 'pool(func)()
@pool
def data(ID, start):
    for i in xrange(start, start+4):
        yield ID, i
        time.sleep(1)

# each of these calls will spawn a thread and return immediately
# make sure you do either pool.finish() or pool.iter()
# otherwise your program will exit before the threads finish
data("generator 1", 5)
data("generator 2", 10)
data("generator 3", 64)

for value in pool.iter():
    # this will print the generators' return values as they yield
    print value