使用tqdm的asyncio aiohttp进度条

Question

我正在尝试集成一个tqdm进度条来监视aiohttp在Python 3.5中生成的POST请求.我有一个工作进度条,但似乎无法使用收集结果as_completed().指针感激不尽.

我发现的示例建议使用以下模式,该模式与Python 3.5 async def定义不兼容:

for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(coros)):
    yield from f

在没有进度条的情况下工作(虽然部分编辑)异步代码:

def async_classify(records):

    async def fetch(session, name, sequence):
        url = 'https://app.example.com/api/v0/search'
        payload = {'sequence': str(sequence)}
        async with session.post(url, data=payload) as response:
            return name, await response.json()

    async def loop():
        auth = aiohttp.BasicAuth(api_key)
        conn = aiohttp.TCPConnector(limit=100)
        with aiohttp.ClientSession(auth=auth, connector=conn) as session:
            tasks = [fetch(session, record.id, record.seq) for record in records]
            responses = await asyncio.gather(*tasks)    
        return OrderedDict(responses)

这是我修改的失败尝试loop():

async def loop():
    auth = aiohttp.BasicAuth(api_key)
    conn = aiohttp.TCPConnector(limit=100)
    with aiohttp.ClientSession(auth=auth, connector=conn) as session:
        tasks = [fetch(session, record.id, record.seq) for record in records]
        for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(tasks)):
            await f
        responses = await asyncio.gather(f)
        print(responses)

Answer 1

await f返回一个单个响应.为什么你会传递一个已经完成Future到asyncio.gather(f)目前还不清楚.

尝试:

responses = []
for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(tasks)):
    responses.append(await f)

Python 3.6实现了PEP 530 - 异步理解:

responses = [await f
             for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(tasks))]

它async def现在在函数内部工作.