多处理池的自动终止进程和子进程

Sha*_*shi 9 python timeout process multiprocessing

我正在使用多处理模块进行并行处理.Bellow代码片段在X位置搜索字符串文件名并返回找到字符串的文件名.但在某些情况下,搜索过程需要很长时间,所以我试图用超过300秒的时间来终止搜索过程.为此我使用超时== 300,如下所示,这会杀死搜索过程,但它会杀死孩子过程由波纹管代码产生.

我试图找到多种方式,但没有成功:/

如何从Pool中删除父进程及其子进程?

import os
from multiprocessing import Pool

def runCmd(cmd):
     lresult = os.popen(cmd).read()
     return lresult

main ():
     p = Pool(4)
     data_paths = [list of paths of store data]
     search_cmds = [ "SearchText.exe %s < %s"%(data_path, filename) for data_path in data_paths ]
     results = [p.apply_async(runCmd, (cmd,), callback = log_result) for cmd in search_cmds]
     try:
        for result in results:
            root.append(result.get(timeout=300))
        #rool holds the result of search process
     except TimeoutError:
        for c in multiprocessing.active_children():
            print '----->',c.pid
            os.kill(c.pid, signal.SIGTERM)
     p.close()
     p.join()

if __name__ == '__main__':
    main()
Run Code Online (Sandbox Code Playgroud)

Process Explorer中的Process Tree:

cmd.exe
------python.exe
----------------python.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
----------------python.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
----------------python.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
----------------python.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
Run Code Online (Sandbox Code Playgroud)

上面的代码片段dosnt杀死子进程

--------------------------cmd.exe
---------------------------------SearchText.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
Run Code Online (Sandbox Code Playgroud)

论文子搜索过程保留,这些子进程也被杀死.

请行会.

谢谢

Sha*_*shi 7

我能够使用psutil模块解决我的问题

在波纹管柱上找到解决方案:

import psutil, os

def kill_proc_tree(pid, including_parent=True):    
    parent = psutil.Process(pid)
    for child in parent.get_children(recursive=True):
        child.kill()
    if including_parent:
        parent.kill()

me = os.getpid()
kill_proc_tree(me)
Run Code Online (Sandbox Code Playgroud)

/sf/answers/296058311/

  • get_children不再存在,该方法称为[children](http://pythonhosted.org/psutil/#psutil.Process.children) (3认同)