Asyncio中的请求 - 关键字参数

Question

我正在使用asyncio与requests模块发出异步HTTP请求.

我可以这样做一个GET请求:

@asyncio.coroutine
def do_checks():
    loop = asyncio.get_event_loop()
    req = loop.run_in_executor(None, requests.get, 'https://api.github.com/user')
    resp = yield from req
    print(resp.status_code)
loop = asyncio.get_event_loop()
loop.run_until_complete(do_checks())

但是,我需要在请求中支持Basic HTTP Auth(此处描述).

根据文档,url和auth都是requests.get()的命名参数.

但是,如果我运行它(请注意添加url =''和auth =''):

@asyncio.coroutine
def do_checks():
    loop = asyncio.get_event_loop()
    req = loop.run_in_executor(None, requests.get, url='https://api.github.com/user', auth=HTTPBasicAuth('user', 'pass'))
    resp = yield from req
    print(resp.status_code)
loop = asyncio.get_event_loop()
loop.run_until_complete(do_checks())

我收到此错误:

TypeError: run_in_executor() got an unexpected keyword argument 'url'

在asyncio.run_in_executor()的原型中,支持其他参数:

BaseEventLoop.run_in_executor(executor, callback, *args)

requests.get()显然支持命名参数(get,auth等).怎么了？

Answer 1

两种方法.创建包装函数,或者只使用会话来提供auth.

使用会话:

@asyncio.coroutine
def do_checks():
    loop = asyncio.get_event_loop()
    session = requests.Session()
    session.auth = HTTPBasicAuth('user', 'pass')
    req = loop.run_in_executor(None, session.get, 'https://api.github.com/user')
    resp = yield from req
    print(resp.status_code)

编写一个包装器函数(注意我在def这里使用的是清晰的,但lambda显然也可以使用):

@asyncio.coroutine
def do_checks():
    def do_req():
        return requests.get('https://api.github.com/user', auth=HTTPBasicAuth('user', 'pass'))
    loop = asyncio.get_event_loop()
    req = loop.run_in_executor(None, do_req)
    resp = yield from req
    print(resp.status_code)

Answer 2

这实际上是一个设计决定asyncio.从asyncio/base_events.py的docstring:

"""Base implementation of event loop.

The event loop can be broken up into a multiplexer (the part
responsible for notifying us of IO events) and the event loop proper,
which wraps a multiplexer with functionality for scheduling callbacks,
immediately or at a given time in the future.

Whenever a public API takes a callback, subsequent positional
arguments will be passed to the callback if/when it is called.  This
avoids the proliferation of trivial lambdas implementing closures.
Keyword arguments for the callback are not supported; this is a
conscious design decision, leaving the door open for keyword arguments
to modify the meaning of the API call itself.
"""

注意那里的最后一句话.

该ASYNCIO PEP注意到这个问题,以及,并建议一个lambda来解决它:

此约定特别不支持关键字参数.关键字参数用于传递有关回调的可选附加信息.这允许API的优雅演变,而不必担心关键字对某个地方的被调用者是否重要.如果必须使用关键字参数调用回调,则可以使用lambda.例如:

loop.call_soon(lambda: foo('abc', repeat=42))