在Scrapy中选择具有非ASCII字符的节点

Question

我有以下简单的web scraper Scrapy:

#!/usr/bin/env python
# -*- coding: latin-1 -*-

from scrapy.http import Request
from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector

class MySpiderTest(BaseSpider):
    name = 'MySpiderTest'
    allowed_domains = ["boliga.dk"]
    start_urls = ["http://www.boliga.dk/bbrinfo/3B71489C-AEA0-44CA-A0B2-7BD909B35618",]

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        item = bbrItem()
        print hxs.select("id('unitControl')/div[2]/table/tbody/tr[td//text()[contains(.,'Antal Badeværelser')]]/td[2]/text()").extract()

但是当我运行蜘蛛时,我得到以下语法错误:

SyntaxError: Non-ASCII character '\xe6' in file... on line 32, but no encoding declared

因为æ在xpath.该XPath是在工作Xpath Checker的Firefox.我尝试过URL编码æ,但是没有用.我错过了什么？

谢谢!

更新:我在代码的开头添加了编码声明(Latin-1应该支持丹麦字符)

Answer 1

为您的XPath表达式使用unicode字符串

hxs.select(u"id('unitControl')/div[2]/table/tbody/tr[td//text()[contains(.,'Antal Badeværelser')]]/td[2]/text()").extract()

要么

hxs.select(u"id('unitControl')/div[2]/table/tbody/tr[td//text()[contains(.,'Antal Badev\u00e6relser')]]/td[2]/text()").extract()

请参阅Python源代码中的Unicode文字