登录站点的 asyncio post 请求

Question

我目前正在编写一个脚本，该脚本首先使用 cfscrape 绕过 cloudflare，然后使用有效负载发出 2 个发布请求以登录该站点。我在 future1 和 future2 帖子中遇到一些错误。这是我的代码：

import asyncio
import requests
import cfscrape

async def main():
s = requests.Session()
s.get('https://www.off---white.com/en/IT')

headers = {
    'Referer': 'https://www.off---white.com/it/IT/login',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'
    }

payload1 = {
    'spree_user[email]': 'email',
    'spree_user[password]': 'password',
    'spree_user[remember_me]': '0',
}

payload2 = {
    'spree_user[email]': 'email',
    'spree_user[password]': 'password',
    'spree_user[remember_me]': '0',
}

scraper = cfscrape.create_scraper(s)
scraper.get('https://www.off---white.com/en/IT', headers=headers)
print('Done')

loop = asyncio.get_event_loop()
print('Starting loop')

future1 = loop.run_in_executor(None, requests.post ,'https://www.off---white.com/it/IT/login', data=payload1, headers=headers)
future2 = loop.run_in_executor(None, requests.post ,'https://www.off---white.com/it/IT/login', data=payload2, headers=headers)
response1 = await future1
response2 = await future2
print(response1.text)
print(response2.text)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

错误：

File "async_amatriciana.py", line 41, in <module>
loop.run_until_complete(main())
File "lib/python3.6/asyncio/base_events.py" line 468, in 
run_until_complete
return future.result()
File "async_amatriciana.py", line 33, in main
future1 = loop.run_in_executor(None, requests.post ,'https://www.off--- 
white.com/it/IT/login', data=payload1, headers=headers)
TypeError: run_in_executor() got an unexpected keyword argument 'data'

Answer 1

\n

BaseEventLoop.run_in_executor（执行器，回调，*args）

\n

\n\n

我运行了你的代码并出现了很多错误，因此我重写了你的代码。你需要知道遵循这些

\n\n

\n
1. 用于cfscrape发布数据而不是requests，除非您将 cookie 添加到发布请求中
\n
2. await必须在里面async def
\n
3. run_in_executor只得到args不kwargs
\n
4. 规则# 9：不要requests在异步代码中使用\xe2\x80\x99t——来自@Brad Solomon
\n

\n\n

重写的代码

\n\n

import asyncio\nimport requests\nimport cfscrape\n\nheaders = {\n    'Referer': 'https://www.off---white.com/it/IT/login',\n    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'\n    }\n\npayload1 = {\n    'spree_user[email]': 'email',\n    'spree_user[password]': 'password',\n    'spree_user[remember_me]': '0',\n}\n\npayload2 = {\n    'spree_user[email]': 'email',\n    'spree_user[password]': 'password',\n    'spree_user[remember_me]': '0',\n}\n\n\ndef post(dict):\n    scraper = cfscrape.create_scraper(requests.Session())\n    req = scraper.post(**dict)\n    return req\n\nasync def get_data():\n    datas = [dict(url='https://www.off---white.com/it/IT/login', data=payload1, headers=headers),\n            dict(url='https://www.off---white.com/it/IT/login', data=payload2, headers=headers)]\n    loop = asyncio.get_event_loop()\n    response = [loop.run_in_executor(None, post , data) for data in datas]\n    result = await asyncio.gather(*response)\n    print(result)\n\n\nloop = asyncio.get_event_loop()\nloop.run_until_complete(get_data())\n

\n