我的 Scrapy 代码不起作用,我不知道为什么。我的蜘蛛是在 Reddit 上爬取权力的游戏 subreddit 的测试。
这是我的代码:
import scrapy
class Redditbot2Spider(scrapy.Spider):
name = 'redditbot2'
allowed_domains = ['www.reddit.com']
start_urls = ['https://www.reddit.com/r/gameofthrones/']
def parse(self, response):
titles = response.selector.xpath('//h2/text()').extract()
votes = response.selector.xpath('//div[@class="_1rZYMD_4xY3gRcSS3p80D0"]/test()').extract()
time = response.selector.xpath('//a[@class="_3jOxDPIQ0KaOWpzvSQo-1s"]/text()').extract()
comments = response.selector.xpath('//span[@class="FHCV02u6Cp2zYL0fhQPsO"])/text()').extract()
for item in zip(titles, votes, time, comments):
scraped_info = {
'title': titles[0],
'vote': votes[1],
'time': time[2],
'comments': comments[3],
}
yield scraped_info
Run Code Online (Sandbox Code Playgroud)
这是错误的日志:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/Library/Python/2.7/site-packages/scrapy/spiders/__init__.py", line 90, in parse …
Run Code Online (Sandbox Code Playgroud)