连接使用asyncio.subprocess.create_subprocess_exec()启动的两个进程

Question

使用旧学校subprocess.Popen()API 启动两个进程时,我可以轻松地将标准从一个进程连接到另一个进程的标准进程,创建一个管道,就像UNIX shell在连接命令时所做的那样|:

from subprocess import Popen, PIPE

process_1 = Popen(['ls'], stdout = PIPE)
process_2 = Popen(['wc'], stdin = process_1.stdout)

process_1.wait()
process_2.wait()

在使用asyncio.subprocess.create_subprocess_exec()(或类似的)异步API时,如何实现相同的目标？这是我试过的:

from asyncio.events import get_event_loop
from asyncio.subprocess import PIPE, create_subprocess_exec

async def main():
    process_1 = await create_subprocess_exec('ls', stdout = PIPE)
    process_2 = await create_subprocess_exec('wc', stdin = process_1.stdout)

    await process_1.wait()
    await process_2.wait()

get_event_loop().run_until_complete(main())

但第二次打电话create_subprocess_exec()抱怨传递的论点stdin没有fileno(这是真的):

Traceback (most recent call last):
  File ".../test-async.py", line 11, in <module>
     get_event_loop().run_until_complete(main())
[...]
  File ".../test-async.py", line 6, in main
     process_2 = await create_subprocess_exec('wc', stdin = process_1.stdout)
[...]
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py", line 1388, in _get_handles
     p2cread = stdin.fileno()
AttributeError: 'StreamReader' object has no attribute 'fileno'

如何获得与上述同步示例相同的结果？

Answer 1

在asyncio中,process.stdout实际上是StreamReader,而不是文件对象.可以通过访问文件对象process._transport._proc.stdout.不幸的是,您将无法使用它,因为它已经在事件循环中注册以提供流接口process.stdout.

解决此问题的一种方法是创建自己的管道并将文件描述符传递给子进程:

async def main():
    read, write = os.pipe()
    process_1 = await create_subprocess_exec('ls', stdout=write)
    os.close(write)
    process_2 = await create_subprocess_exec('wc', stdin=read, stdout=PIPE)
    os.close(read)
    return await process_2.stdout.read()

请注意,write一旦启动第一个子进程,应该显式关闭文件描述符(除非您使用它,否则不会自动关闭subprocess.PIPE).该read文件描述符还需要关闭,如解释在这里.