标签: aiohttp

我aiohttp用来请求网址.大部分时间它正常运行,但有时它会停止而不会引发任何异常.

正如您在代码中看到的,我捕获了所有异常,但是当它停止时,不会打印异常日志.

日志看起来像:

get_live_league_games: while True
try
yield from aiohttp.request

但' res = yield from r.json()'不打印,它停止并且不会抛出任何异常.

while True:
    print('get_live_league_games: while True')
    start = time.clock()
    try:
        print('try')
        r = yield from aiohttp.request('GET',url)
        print('yield from aiohttp.request')
        res = yield from r.json()
        print('res = yield from r.json()')
    except aiohttp.errors.DisconnectedError as e:
        logging.warning('get_live_league_games:',e)
        yield from asyncio.sleep(10)
        continue
    except aiohttp.errors.ClientError as e:
        logging.warning('get_live_league_games:',e)
        yield from asyncio.sleep(10)
        continue
    except aiohttp.errors.HttpProcessingError as e:
         logging.warning('get_live_league_games:',e)
         yield from asyncio.sleep(10)
         continue
    except Exception as e:
         logging.warning('get_live_league_games,Exception:',e)
         yield …

我正在尝试创建一个程序,以便与我创建的服务器建立大量的Web套接字连接:

class WebSocketClient():

    @asyncio.coroutine
    def run(self):
        print(self.client_id, 'Connecting')
        ws = yield from aiohttp.ws_connect(self.url)
        print(self.client_id, 'Connected')
        print(self.client_id, 'Sending the message')
        ws.send_str(self.make_new_message())

        while not ws.closed:
            msg = yield from ws.receive()

            if msg.tp == aiohttp.MsgType.text:
                print(self.client_id, 'Received the echo')
                yield from ws.close()
                break

        print(self.client_id, 'Closed')


@asyncio.coroutine
def make_clients():

    for client_id in range(args.clients):
        yield from WebSocketClient(client_id, WS_CHANNEL_URL.format(client_id=client_id)).run()


event_loop.run_until_complete(make_clients())

问题是所有客户一个接一个地完成工作:

0 Connecting
0 Connected
0 Sending the message
0 Received the echo
0 Closed
1 Connecting
1 Connected
1 Sending the message
1 Received the …

我有两个脚本,scraper.py和db_control.py.在scraper.py我有这样的事情:

...
def scrap(category, field, pages, search, use_proxy, proxy_file):
    ...
    loop = asyncio.get_event_loop()

    to_do = [ get_pages(url, params, conngen) for url in urls ]
    wait_coro = asyncio.wait(to_do)
    res, _ = loop.run_until_complete(wait_coro)
    ...
    loop.close()

    return [ x.result() for x in res ]

...

在db_control.py中:

from scraper import scrap
...
while new < 15:
    data = scrap(category, field, pages, search, use_proxy, proxy_file)
    ...
...

从理论上讲,刮板应该在未知时间开始,直到获得足够的数据.但是当new不是imidiatelly > 15然后这个错误发生:

  File "/usr/lib/python3.4/asyncio/base_events.py", line 293, in run_until_complete
self._check_closed()
  File "/usr/lib/python3.4/asyncio/base_events.py", line 265, in …

我正在编写应用程序,它每秒扫描目录,检查新文件,如果它们出现 - 通过POST请求发送它们并执行存档.假设可以出现在目录中的文件数量可以从10到100 - 我决定使用asyncio和aiohttp来同时发送请求.

码:

import os
import aiohttp
from aiohttp.client import ClientSession

BASE_DIR = '/path/to'
ARCHIVE_DIR = '/path/to/archive'

async def scan():
    while True:
        await asyncio.sleep(1)
        for file in os.listdir(BASE_DIR):
            if os.path.join(BASE_DIR, file).endswith('jpg'):
                asyncio.ensure_future(publish_file(file))


async def publish_file(file):
    async with ClientSession(loop=loop) as session:
        async with session.post(url=url, data={'photo': open(os.path.join(BASE_DIR, file), 'rb')}) as response:
            if response.status == 200:
                await move_to_archive(file)

async def move_to_archive(file):
    os.rename(os.path.join(BASE_DIR, file), os.path.join(ARCHIVE_DIR, file))

loop = asyncio.get_event_loop()

coros = [
    asyncio.ensure_future(scan())
]
loop.run_until_complete(asyncio.wait(coros))

所以,问题是:如果我要发送的请求的并发,这是一个很好的做法,协同程序添加到循环是这样的:asyncio.ensure_future(publish_file(file))？

我最近从flask+ requests上移到了aiohttp它的async http客户端.

在我的场景中,我需要调用API HTTPS(使用自定义证书)并发送客户端证书.

对于第一部分(验证自定义证书),文档中的支持清晰明确,并且工作得很好.

另一方面,对于第二部分,我似乎无法找到一种简单的方法来附加自定义SSL客户端证书来授权客户端.

你们知道怎么做吗？非常感谢 !

我正在尝试使用Gunicorn运行基于aiohttp的服务器.

这是命令:

gunicorn aiohttpdemo_polls:app --bind 127.0.0.1:8080

它返回:

Failed to find application: 'aiohttpdemo_polls' 

但是当我使用python -m运行它时,如下所示:

python -m aiohttpdemo_polls

它工作正常.代码可以在这里找到,这是aiohttp repo中的一个演示应用程序.也尝试过如下:

gunicorn aiohttpdemo_polls.main:app --bind 127.0.0.1:8080

但它也没有运行服务器.它回来了

Failed to find application: 'aiohttpdemo_polls.main'

知道在哪里进一步寻找解决问题的方法吗？

我是aiohttp的新手，只是在测试它的某些功能。因此，我创建了一个简单的客户端和服务器脚本，以查看异步代码可以处理多少个请求，但是我遇到了错误。

服务器脚本如下：

from aiohttp import web

# Initialize counter
counter = 1

# View index function
async def index(request):
    # Increments counter
    global counter
    counter += 1

    # Return data
    data = {'test': counter}
    return web.json_response(data)

# Creating application
app = web.Application()

# Registering route
app.router.add_get('/', index)

# Starting server
web.run_app(app, host='127.0.0.1', port=8080)

客户端脚本如下：

import asyncio, time, aiohttp
from aiohttp import ClientSession

# Initialize counter
COUNT = 0
requests = 2

async def fetch(url, session):
    async with session.get(url) as response: …

尝试使用信号量来控制异步请求，以控制对目标主机的请求，但是出现以下错误（我已经假设这asycio.sleep()是错误的），即我实际上并未处于睡眠状态。我怎样才能解决这个问题？我想在每个目标网址的请求中添加延迟。

错误：

RuntimeWarning: coroutine 'sleep' was never awaited
Coroutine created at (most recent call last)
  File "sephora_scraper.py", line 71, in <module>
    loop.run_until_complete(main())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 571, in run_until_complete
    self.run_forever()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 539, in run_forever
    self._run_once()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 1767, in _run_once
    handle._run()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "makeup.py", line 26, in get_html
    asyncio.sleep(delay)
  asyncio.sleep(delay)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

码：

import sys
import time
import asyncio
import aiohttp

async …

我最近遇到了Ellipsis（...），它在aiohttp代码中的函数参数中使用，然后在该函数的主体中使用：

def make_mocked_request(method, path, headers=None, *,
                        match_info=sentinel,
                        version=HttpVersion(1, 1), closing=False,
                        app=None,
                        writer=sentinel,
                        protocol=sentinel,
                        transport=sentinel,
                        payload=sentinel,
                        sslcontext=None,
                        client_max_size=1024**2,
                        loop=...):
    """Creates mocked web.Request testing purposes.

    Useful in unit tests, when spinning full web server is overkill or
    specific conditions and errors are hard to trigger.

    """

    task = mock.Mock()
    if loop is ...:
        loop = mock.Mock()
        loop.create_future.return_value = ()

您能解释这个新的python 3功能吗？

我的问题是关于做出回应的正确方式 aiohttp

官方 aiohttp 文档为我们提供了进行异步查询的示例：

session = aiohttp.ClientSession()

async with session.get('http://httpbin.org/get') as resp:
    print(resp.status)
    print(await resp.text())

await session.close()

我不明白，为什么这里的上下文管理器。我发现的只是__aexit__()方法等待resp.release()方法。但是文档还告诉我们resp.release()一般不需要等待。

这一切真的让我很困惑。

如果我发现下面的代码更具可读性并且不是那么嵌套，为什么要这样做？

session = aiohttp.ClientSession()

resp = await session.get('http://httpbin.org/get')
print(resp.status)
print(await resp.text())

# I finally have not get the essence of this method.
# I've tried both using and not using this method in my code,
# I've not found any difference in behaviour.
# await resp.release()

await session.close()

我已经深入研究了aiohttp.ClientSession它的上下文管理器来源，但我没有发现任何可以澄清情况的东西。

最后，我的问题是：有什么区别？