使用subprocess.Popen扭曲线程化？

Question

我正在尝试使用Twisted实现一个服务,它与这里的"finger"教程非常接近:http://twistedmatrix.com/documents/current/core/howto/tutorial/intro.html

我有一个basic.LineListener等待命令然后执行它,然后我有一个客户端连接并发出命令.麻烦的是命令有时需要执行其他操作,而我正在使用python的子进程模块.它不仅仅是对来自()传递的调用,这是一个正常的子进程问题,我知道如何通过它.这是subprocess.Popen调用挂起.

这是扭曲的服务器代码:

from twisted.application import internet, service
from twisted.internet import protocol, reactor, defer, threads
from twisted.protocols import basic
import sys
import time
import subprocess

class MyProtocol(basic.LineReceiver):
    def lineReceived(self, line):
        self.go()
    def go(self):
        def writeResponse(message):
            self.transport.write(message + '\r\n')
            self.transport.loseConnection()
        threads.deferToThread(self.factory.action).addCallback(writeResponse)
    def connectionMade(self):
        self.lines = []

class ActionService(service.Service):
    def __init__(self, **kwargs):
        pass
        #self.users = kwargs

def action(self):
    print "launching subprocess"
    sys.stdout.flush()
    p = subprocess.Popen(["ls"], stderr=subprocess.PIPE, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
    print "launched subprocess, trying to communicate..."
    sys.stdout.flush()
    p.communicate()
    print "returning"
    sys.stdout.flush()
    return "%032d" % (0)

    def getActionFactory(self):
        f = protocol.ServerFactory()
        f.protocol = MyProtocol
        f.action = self.action
        return f

reactor.suggestThreadPoolSize(300)
application = service.Application('Action', uid=0, gid=0)
f = ActionService()
serviceCollection = service.IServiceCollection(application)
internet.TCPServer(31337,f.getActionFactory()
                   ).setServiceParent(serviceCollection)

...这里有一些客户端代码:

#!/usr/bin/python
import time
import threading
import socket

def connectAction(host):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((host, 31337))
    s.send("asdf\r\n")
    resp = s.recv(32)
    s.close()
    return resp

class sscceThread(threading.Thread):
    def __init__(self, host):
        self.host = host
        threading.Thread.__init__(self)
    def run(self):
        connectAction(self.host)

def main():
    threads = []
    for i in range(0, 1000):
        for j in range(0,5):
            t = sscceThread("localhost")
            t.start()
            threads.append(t)
        for t in threads:
            t.join()
        print i
        time.sleep(1)
    #    print i

if __name__ == "__main__":
    main()

运行以下命令启动服务:

twistd -y sscce_twisted_service.py -l twistdLog; tail -f twistdLog

并通过运行来运行客户端:

./sscce_twisted_client.py

您应该看到客户端进行了几次迭代(我已经看到它多达10次)然后挂起.客户端代码包含1秒的睡眠时间,这样您就可以区分每次迭代中的扭曲日志条目与挂起的日志条目之间的区别,您将在扭曲的日志中看到类似的内容:

2009-12-22 11:18:47-0800 [MyProtocol,55,127.0.0.1] launching subprocess
2009-12-22 11:18:47-0800 [MyProtocol,56,127.0.0.1] launching subprocess
2009-12-22 11:18:47-0800 [MyProtocol,55,127.0.0.1] launched subprocess, trying to communicate...
2009-12-22 11:18:47-0800 [MyProtocol,57,127.0.0.1]  launching subprocess
2009-12-22 11:18:47-0800 [MyProtocol,58,127.0.0.1] launching subprocess
2009-12-22 11:18:47-0800 [MyProtocol,56,127.0.0.1] launched subprocess, trying to communicate...
2009-12-22 11:18:47-0800 [MyProtocol,55,127.0.0.1] returning
2009-12-22 11:18:47-0800 [MyProtocol,57,127.0.0.1]  launching subprocess 
2009-12-22 11:18:47-0800 [MyProtocol,56,127.0.0.1]  launching subprocess returning
2009-12-22 11:18:47-0800 [MyProtocol,59,127.0.0.1]  launching subprocess
2009-12-22 11:18:47-0800 [MyProtocol,58,127.0.0.1]  launched subprocess, trying to communicate...
2009-12-22 11:18:47-0800 [MyProtocol,58,127.0.0.1] returning
2009-12-22 11:18:47-0800 [MyProtocol,59,127.0.0.1] launched subprocess, trying to communicate...
2009-12-22 11:18:47-0800 [MyProtocol,59,127.0.0.1] returning

特别值得注意的是MyProtocol,57.它说它即将尝试启动子流程,但它从未打印过"启动的子流程,试图通信"这一行.我认为它必须挂在那里.

Answer 1

正如mg在评论中所说,不要使用子进程模块.在POSIX平台上,有必要(或多或少)处理SIGCHLD信号以处理退出的子进程.由于只能有一个SIGCHLD处理程序,因此多个库通常不会合作.Twisted的子进程支持和子进程模块的支持冲突.要么使用Twisted的支持(请参阅http://twistedmatrix.com/documents/current/core/howto/process.html),要么通过传递installSignalHandlers=False来禁用Twisted的支持reactor.run(我推荐前者,因为subprocess提供的阻塞接口不能很好地集成到基于扭曲的应用程序).