Asyncio使HTTP请求更慢？

Question

我正在使用Asyncio和Requests来对一系列HTTP请求进行基准测试.

出于某种原因,使用Asyncio比使用直接请求稍慢.知道为什么吗？我是否错误地使用Asyncio？

import asyncio
import functools
import requests
import time

ts = time.time()
for i in range(10):
  @asyncio.coroutine
  def do_checks():
      loop = asyncio.get_event_loop()
      req = loop.run_in_executor(None, functools.partial(requests.get, "http://google.com", timeout=3))
      resp = yield from req
      print(resp.status_code)

  loop = asyncio.get_event_loop()
  loop.run_until_complete(do_checks())
te = time.time()
print("Version A: " + str(te - ts))

ts = time.time()
for i in range(10):
  r = requests.get("http://google.com", timeout=3)
  print(r.status_code)
te = time.time()

print("Version B:  " + str(te - ts))

输出:

版本A = Asyncio; 版本B =请求

200
200
200
200
200
200
200
200
200
200
Version A: 5.7215821743011475
200
200
200
200
200
200
200
200
200
200
Version B:  5.320340156555176

Answer 1

在开始下一个请求之前,您正在等待每个请求完成.所以你有事件循环的开销没有任何好处.

试试这个:

import asyncio
import functools
import requests
import time

ts = time.time()
loop = asyncio.get_event_loop()

@asyncio.coroutine
def do_checks():
    futures = []
    for i in range(10):
        futures.append(loop.run_in_executor(None, functools.partial(requests.get, "http://google.com", timeout=3)))

    for req in asyncio.as_completed(futures):
        resp = yield from req
        print(resp.status_code)

loop.run_until_complete(do_checks())
te = time.time()
print("Version A: " + str(te - ts))

ts = time.time()
for i in range(10):
    r = requests.get("http://google.com", timeout=3)
    print(r.status_code)
te = time.time()
print("Version B:  " + str(te - ts))

这是我运行时得到的:

$ python test.py 
200
...
Version A: 0.43438172340393066
200
...
Version B: 1.6541109085083008

速度要快得多,但实际上这只是产生线程并等待http库完成,你不需asyncio要这样做.

您可能希望结帐,aiohttp因为它是为了使用而构建的asyncio.requests是一个神话般的图书馆,但它不是为了asyncio.

Answer 2

为了完整起见,这是一个非常快速的asyncio实现

import aiohttp
import asyncio
import time

async def main(n):
    ts = time.time()
    session = aiohttp.ClientSession()
    fs = [session.get('http://google.com') for _ in range(n)]
    for f in asyncio.as_completed(fs):
        resp = await f
        print(resp.status)
        await resp.release()
    await session.close()
    te = time.time()
    print("Aiohttp version:  " + str(te - ts))

loop = asyncio.get_event_loop()
loop.run_until_complete(main(10))
loop.close()

代码是python 3.5及更高版本.

~> python asyncioreq.py
200
...
Aiohttp version:  0.15974688529968262

希望有人可以使用它;)