编写并行循环

Question

我试图在一个简单的例子上运行并行循环.
我究竟做错了什么？

from joblib import Parallel, delayed  
import multiprocessing

def processInput(i):  
        return i * i

if __name__ == '__main__':

    # what are your inputs, and what operation do you want to 
    # perform on each input. For example...
    inputs = range(1000000)      

    num_cores = multiprocessing.cpu_count()

    results = Parallel(n_jobs=4)(delayed(processInput)(i) for i in inputs) 

    print(results)

代码的问题在于,当在Python 3中的Windows环境下执行时,它会打开num_corespython实例来执行并行作业,但只有一个是活动的.这不应该是这种情况,因为处理器的活动应该是100%而不是14%(在i7-8个逻辑核心下).

为什么额外的实例没有做任何事情？

Answer 1

继续请求提供有效的多处理代码,我建议你使用pool_map(如果延迟功能并不重要),我会给你一个例子,如果你正在使用python3,值得一提的是你可以使用starmap .另外值得一提的是,如果返回结果的顺序不必与输入顺序相对应,则可以使用map_sync/starmap_async.

import multiprocessing as mp

def processInput(i):
        return i * i

if __name__ == '__main__':

    # what are your inputs, and what operation do you want to
    # perform on each input. For example...
    inputs = range(1000000)
    #  removing processes argument makes the code run on all available cores
    pool = mp.Pool(processes=4)
    results = pool.map(processInput, inputs)
    print(results)