在多进程模块中运行子进程的顺序

Question

使用for循环启动多进程.

import os
from multiprocessing import Process

def run_proc(name):
    print('child process %s (%s) running ...' %(name,os.getpid()))

if __name__ == '__main__':
    print('parent process %s.' %os.getppid())
    for i in range(5):
        p = Process(target=run_proc,args=(str(i),))
        print('process will start'+str(i))
        p.start()
    p.join()
    print('process is end')

我得到了结果.

parent process 6497.  
process will start  
process will start  
child process 0 (6984) running ...  
process will start  
process will start  
process will start  
child process 2 (6986) running ...  
child process 1 (6985) running ...  
child process 3 (6987) running ...  
child process 4 (6988) running ...  
process is end  

为什么早期创建的子进程稍后执行？
为什么不能得到以下结果？

parent process 6497.  
process will start  
process will start  
child process 0 (6984) running ...  
process will start  
process will start  
process will start  
child process 1 (6986) running ...  
child process 2 (6985) running ...  
child process 3 (6987) running ...  
child process 4 (6988) running ...  
process is end

Jean-FrançoisFabre所说的是如何创建以下结果:

parent process 6497.
process will start
child process 0 (9639) running ...
process will start
child process 1 (9640) running ...
process will start
child process 2 (9641) running ...
process will start
child process 3 (9643) running ...
process will start
child process 4 (9644) running ...
process is end

我只能在for循环中改变p.join,如下所示:

import os
from multiprocessing import Process

def run_proc(name):
    print('child process %s (%s) running ...' %(name,os.getpid()))

if __name__ == '__main__':
    print('parent process %s.' %os.getppid())
    for i in range(5):
        p = Process(target=run_proc,args=(str(i),))
        print('process will start'+str(i))
        p.start()
        p.join()
    print('process is end')

我想知道的是为什么我的代码导致以下输出

parent process 6497.  
process will start  
process will start  
child process 0 (6984) running ...  
process will start  
process will start  
process will start  
child process 2 (6986) running ...  
child process 1 (6985) running ...  
child process 3 (6987) running ...  
child process 4 (6988) running ...  
process is end  

代替:

parent process 6497.  
process will start  
process will start  
child process 0 (6984) running ...  
process will start  
process will start  
process will start  
child process 1 (6986) running ...  
child process 2 (6985) running ...  
child process 3 (6987) running ...  
child process 4 (6988) running ...  
process is end

这是一个不同的问题.

Answer 1

在这里,您在创建流程的主流程和尝试启动的流程(以及子流程本身)之间存在竞争条件.

一旦p.start()从主进程发出命令,子进程就可以运行(并打印).但主要流程也在努力创建下一个子流程.谁将首先打印下一行？很难知道.如果主进程成功创建下一个子进程,那么现在竞争条件是在两个子进程之间:您正在经历的内容.

这些进程可以并行运行,但在调用操作系统时它们仍然具有同步点.首先到达操作系统的人首先获得服务(例如:打印到控制台).

当然,p.join() 在循环内部恢复顺序,它也会取消多处理的效果,因为主进程在创建另一个进程之前等待子进程结束.

它通常无关紧要,因为你正在做一些并行任务.

我首先在列表推导中创建进程,然后循环启动它们,并稍微延迟以确保在创建下一个进程之前启动并打印进程.

process_list = [Process(target=run_proc,args=(str(i),)) for i in range(5)]
for i,p in enumerate(process_list):
    print('process {} will start'.format(i))
    p.start()
    time.sleep(0.1)

当主进程等待一段时间时,这为子进程提供了启动和打印的喘息空间.

另请注意,您的上一个p.join()只是加入循环的最后一个进程,它应该是(现在使用我们全新的process_list):

for p in process_list:
    p.join()

在大多数情况下,开始/结束顺序并不重要.您可以预先计算主流程中的所有订购信息(就像您通过为流程名称分配不断增加的数字一样).

请注意,经典问题可能不是确保进程按顺序启动,而是它们产生的结果可以与您提供的输入匹配(将进程传递给进程,并生成输出列表,因此在你知道哪个输入提供了哪个输出).

在这种情况下,查找multiprocessing.pool和map函数(Python multiprocessing.pool顺序运行的进程)