小编Ric*_*rdo的帖子

Scrapy + Splash = 拒绝连接

我使用此链接安装了Splash。按照所有步骤进行安装,但 Splash 不起作用。

我的settings.py文件:

BOT_NAME = 'Teste'
SPIDER_MODULES = ['Test.spiders']
NEWSPIDER_MODULE = 'Test.spiders'
DOWNLOADER_MIDDLEWARES = {
     'scrapy_splash.SplashCookiesMiddleware': 723,
     'scrapy_splash.SplashMiddleware': 725, 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': 810,}
SPIDER_MIDDLEWARES = {
'scrapy_splash.SplashDeduplicateArgsMiddleware': 100,
}
SPLASH_URL = 'http://127.0.0.1:8050/'
Run Code Online (Sandbox Code Playgroud)

当我运行时scrapy crawl TestSpider

[scrapy.core.engine] INFO: Spider opened
[scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
[scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET http://www.google.com.br via http://127.0.0.1:8050/render.html> (failed 1 times): Connection was refused by other side: 111: Connection refused.
[scrapy.downloadermiddlewares.retry] …
Run Code Online (Sandbox Code Playgroud)

web-crawler scrapy scrapy-splash splash-js-render

2
推荐指数
2
解决办法
2971
查看次数