我是 Scrapy 的新手,我正在尝试抓取该网站https://www.timeanddate.com/weather/india及其引发的DNS 查找错误。我为抓取而编写的代码在 shell 中完美运行,所以我猜测是在抓取发生之前发生了 DNS 错误。
这就是我得到的:
2019-05-02 11:59:03 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: IndiaWeather)
2019-05-02 11:59:03 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.0, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17134-SP0
2019-05-02 11:59:03 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'IndiaWeather', 'NEWSPIDER_MODULE': 'IndiaWeather.spiders', 'SPIDER_MODULES': ['IndiaWeather.spiders']}
2019-05-02 11:59:03 [scrapy.extensions.telnet] INFO: Telnet Password: 688b4fe759cb3ed5
2019-05-02 …Run Code Online (Sandbox Code Playgroud)