保持 aiohttp 会话处于活动状态

Question

我尝试每 X 秒访问一个具有并行和单独会话的网站，然后分析响应中的内容以查看每个会话是否应该继续。但是，一旦代码到达第二个循环，它就会失败。

import asyncio
from aiohttp import ClientSession
import logging
import time

interval = 30
instances = 2
visit_url = 'http://www.example.org'

tasks = []

logging.basicConfig(
    format='%(asctime)s.%(msecs)03d %(message)s',  # Log in format time.milliseconds {message}
    level=logging.INFO,  # Use with logging.info()
    datefmt='%H:%M:%S')  # Display time as Hours:Minutes:Seconds


class StopException(Exception):
    pass


async def quit_app(session, task_, reason):
    logging.info("[{}] {}.".format(task_, reason))
    session.cookies.clear()  # Reset cookies
    session.headers.clear()  # Reset headers
    session.close()  # End HTTP connection
    raise StopException


async def get_status(response):
    if "abow" in response:
        return "success"
    elif "odoap" or "daoscp" in response:
        return "waiting"
    elif "nullt" in response:
        return "fail"
    elif "issue" in response:
        return "banned"
    elif "pending" in response:
        return "pending"
    else:
        return "pending"


async def initialise(headers):
    session = ClientSession()
    task_ = len(asyncio.Task.all_tasks()) - instances - 1
    passed = False
    while passed is False:
        async with session as session:
            async with session.get(visit_url, headers=headers, allow_redirects=True) as initial:
                status = await get_status(await initial.text())  # Check HTML for status
                if status == "success":
                    logging.info("[{}] {}.".format(task_, "Success"))
                    passed = True
                elif status == "pending":
                    logging.info("[{}] {}.".format(task_, "Pending.."))
                    await asyncio.sleep(interval)
                elif status == "waiting":
                    logging.info("[{}] {}.".format(task_, "Waiting..."))
                    await asyncio.sleep(interval)
                elif status == "banned":
                    await quit_app(initial, task_, "Banned")
                elif status == "fail":
                    await quit_app(initial, task_, "Failed")


if __name__ == "__main__":
    headers = {
        'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
        'accept-encoding': 'gzip, deflate, br',
        'accept-language': 'en-US,en;q=0.9',
        'upgrade-insecure-asks': '1',
        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36'
    }  # Add appropriate headers
    start = time.clock()
    loop = asyncio.get_event_loop()
    for i in range(instances):
        task = asyncio.ensure_future(initialise(headers))
        tasks.append(task)
    loop.run_until_complete(asyncio.wait(tasks))
    end = time.clock()
    print("Process took {0:.4f} seconds.".format(end - start))

此代码返回以下错误：

13:56:58.604 未来从未检索到任务异常：任务完成 coro=initialise() 完成，在 C:/Users/x/PycharmProjects/tests/src/aiohttp_main.py:49 中定义 异常 = RuntimeError('会话已关闭' ,) RuntimeError: 会话已关闭

我只是不知道如何让会话保持活动状态，直到我 .close() 它们......

Answer 1

我只是不知道如何让会话保持活动状态，直到我 .close() 它们......

这async with是关闭会话的请求。这意味着你写完之后：

async with session as session:

session...一旦执行完毕，您就不能再使用async with。这并不是 aiohttp 所特有的，而是withPython 中的工作原理。例如，在处理文件时，with请求关闭文件：

with open('data.csv') as fileobj:
    # ... read stuff from fileobj

# outside the "with" block, fileobj is closed and you
# can no longer read from it

修复方法很简单，只需将循环移到with外部即可while。例如：

async def initialise(headers):
    async with ClientSession() as session:
        # the rest of the code, including the `while` loop, here

在一个不相关的说明中，您可能想要替换len(asyncio.Task.all_tasks())为您自己的全局计数器。Task.all_tasks()如果您稍后将其他不相关的任务合并到事件循环中（或者第三方库为您执行此操作），以这种方式使用可能会开始产生不正确的结果。