Aei*_*ius 5 python beautifulsoup cloudflare
所以我试图绕过一个网站的 cloudflare 保护来从他们那里抓取一些项目，但Cloudscraper python 模块不起作用。
每当我运行它时，我都会收到此错误：
cloudscraper.exceptions.CloudflareChallengeError: Detected a Cloudflare version 2 challenge, This feature is not available in the opensource (free) version.
这是我正在使用的简化代码：
import cloudscraper
from bs4 import BeautifulSoup as soup
url = "http://adventurequest.life/"
scraper = cloudscraper.create_scraper()
html = scraper.get(url).text
page_soup = soup(html, "html.parser")
print(page_soup)
你们知道如何解决这个问题吗？
小智 -3
我在使用scrapy + cloudscraper时遇到了同样的错误，但后来我设置了cookie_enable=true就好了\xef\xbc\x9a
\n错误
\nTraceback (most recent call last):\ncloudscraper.exceptions.CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.\n2021-04-27 09:59:30 [scrapy.core.scraper] ERROR: Error downloading <GET https://www.forever21.com/us/shop/catalog/category/f21/lingerie>\nTraceback (most recent call last):\nStopIteration: <403 \nhttps://www.forever21.com/us/shop/catalog/category/f21/lingerie>\n之前\xef\xbc\x9a
\nTraceback (most recent call last):\ncloudscraper.exceptions.CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.\n2021-04-27 09:59:30 [scrapy.core.scraper] ERROR: Error downloading <GET https://www.forever21.com/us/shop/catalog/category/f21/lingerie>\nTraceback (most recent call last):\nStopIteration: <403 \nhttps://www.forever21.com/us/shop/catalog/category/f21/lingerie>\n后：
\n\'COOKIES_ENABLED\': True\n但在 bs4 中默认添加 Cookie\xef\xbc\x8c 所以我尝试了你的代码，发现它是正常的。
\nurl = "http://adventurequest.life/"\nscraper = cloudscraper.create_scraper()\nhtml = scraper.get(url).text\npage_soup = soup(html, "html.parser")\nprint(page_soup)\n\n<!DOCTYPE doctype html>\n<html lang="en" style="min-height: 100%;">\n<head>\n<!-- Required meta tags -->\n<meta charset="utf-8"/>\n<meta content="width=device-width, initial-scale=1, shrink-to-fit=no" name="viewport"/>\n<meta content="Auto Quest Worlds" name="twitter:title"/>\n<meta content="aqw bots, adventure quest bots, aqw cheat, aqw hack, aqw exploits, grimoire download, adventure quest worlds bot, leveling bot aqw, botting mmorpg, aqw private server, aqworlds private server, aqw server, aqw ps, aqw private, skidson, aqw pirata, servidor de aqw, adventure quest worlds private, dragonfable private server, adventure quest private server, free to play mmorpg, free online games, browser games, jogos online, jogos criancas, jogos de navegador, best aqw private server, best online mmorpg, best browser mmorpg, habbo servidor privado, habbo retro, habbo private server, runescape private server, high rates aqw, aqw items, aqworlds wiki" name="keywords"/>\n<meta content="https://adventurequest.life/" name="twitter:url"/>\n也许你应该检查你的机器 opennssl 版本然后更新或升级 cloudcraper 版本。
\n我的cloudscraper版本是\xef\xbc\x9a cloudscraper ========> 1.2.58
\n| 归档时间： |
|
| 查看次数： |
5256 次 |
| 最近记录： |