在上一个问题中,用户建议使用以下方法来获取多个 url(API 调用)aiohttp:
import asyncio
import aiohttp
url_list = ['https://api.pushshift.io/reddit/search/comment/?q=Nestle&size=30&after=1530396000&before=1530436000', 'https://api.pushshift.io/reddit/search/comment/?q=Nestle&size=30&after=1530436000&before=1530476000']
async def fetch(session, url):
async with session.get(url) as response:
return await response.json()['data']
async def fetch_all(session, urls, loop):
results = await asyncio.gather(*[loop.create_task(fetch(session, url)) for url in urls], return_exceptions= True)
return results
if __name__=='__main__':
loop = asyncio.get_event_loop()
urls = url_list
with aiohttp.ClientSession(loop=loop) as session:
htmls = loop.run_until_complete(fetch_all(session, urls, loop))
print(htmls)
Run Code Online (Sandbox Code Playgroud)
但是,这只会导致返回属性错误:
[AttributeError('__aexit__',), AttributeError('__aexit__',)]
Run Code Online (Sandbox Code Playgroud)
(我启用了,否则它就会中断)。我真的希望这里有人可以帮助解决这个问题,仍然很难找到asyncio等的资源。返回的数据是json格式。最后,我想将所有 json 字典放在一个列表中。