来自scrapy.selector导入选择器错误

Question

我无法执行以下操作:

from scrapy.selector import Selector

错误是:

文件"/Desktop/KSL/KSL/spiders/spider.py",第1行,来自scrapy.selector import Selector ImportError:无法导入名称Selector

好像LXML没有安装在我的机器上,但确实如此.另外,我认为这是scrapy内置的默认模块.也许不吧？

思考？

Answer 1

请尝试导入HtmlXPathSelector.

    from scrapy.selector import HtmlXPathSelector

然后使用.select()方法解析出你的html.例如,

    sel = HtmlXPathSelector(response)
    site_names = sel.select('//ul/li')

如果您正在关注Scrapy站点(http://doc.scrapy.org/en/latest/intro/tutorial.html)上的教程,则更新后的示例如下所示:

    from scrapy.spider import BaseSpider
    from scrapy.selector import HtmlXPathSelector

    class DmozSpider(BaseSpider):
        name = "dmoz"
        allowed_domains = ["dmoz.org"]
        start_urls = [
            "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
            "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
        ]

        def parse(self, response):
            sel = HtmlXPathSelector(response)
            sites = sel.select('//ul/li')

            for site in sites:
                title = site.select('a/text()').extract()
                link = site.select('a/@href').extract()
                desc = site.select('text()').extract()
                print title, link, desc

希望这可以帮助!