Python 3并发.futures - 并行处理for循环

Question

如果我正确理解了concurrent.futuresPython 3 中的模块是如何工作的，则以下代码：

import concurrent.futures
import threading

# Simple function returning a value
def test(i):

    a = 'Hello World\n'
    return a


def main():
    output1 = list()

    with concurrent.futures.ThreadPoolExecutor() as executor:

        # psdd iterator to test function
        for out1 in executor.map(test, range(0, 10)):
            # append returned result
            output1.append(out1)

            # confirm output
            print(output1)
            print("Task Executed {}".format(threading.current_thread()))


if __name__ == '__main__':
    main()

...执行以下功能：

1. 将 for 循环传递给名为 的函数test()。
2. 并行处理循环，而不是串行处理。

然而，我真正想要的是在我的函数中并行处理循环，main()如下所示：

import concurrent.futures
import threading


def main():
    output1 = list()

    with concurrent.futures.ThreadPoolExecutor() as executor:

        # psdd iterator to test function
        for out1 in executor.submit(range(0, 10)):

            a = 'Hello World\n'
            # append returned result
            output1.append(a)

            # confirm output
            print(output1)
            print("Task Executed {}".format(threading.current_thread()))


if __name__ == '__main__':
    main()

...但这会产生以下错误：

Traceback (most recent call last):
  File "G:\HTPC Scripts\WebGrab Plus\TESTTESTTEST2.py", line 221, in <module>
    main()
  File "G:\HTPC Scripts\WebGrab Plus\TESTTESTTEST2.py", line 209, in main
    for out1 in executor.submit(range(0, 10)):
TypeError: 'Future' object is not iterable

我的代码需要修改什么？

谢谢

Answer 1

无论您使用submitor map，您始终必须使用可调用对象（例如函数）作为第一个参数。

Python确实允许嵌套函数（还要注意使用方式Futures）；

import concurrent.futures


def main():

    def worker(arg):
        return str(arg) + ' Hello World!'

    with concurrent.futures.ThreadPoolExecutor() as e:
        fut = [e.submit(worker, i) for i in range(10)]
        for r in concurrent.futures.as_completed(fut):
            print(r.result())


if __name__ == '__main__':
    main()

就地定义可调用的唯一方法是使用表达式lambda，但这些方法有很大的局限性。