Ray*_*Luo 1 html python parsing entity elementtree
有时您想要解析一些格式合理的HTML页面,但是您不愿意引入额外的库依赖项,例如BeautifulSoup或lxml.因此,您可能希望首先尝试内置的ElementTree,因为它是一个标准库,它很快(在C中实现),并且它支持比基本HTMLParser更好的接口(例如XPATH支持).更不用说,HTMLParser有其自身的局限性.
ElementTree将工作,直到它遇到某些实体,例如 ,默认情况下不处理的实体.
import xml.etree.ElementTree as ET
html = '''<html>
<div>Some reasonably well-formed HTML content.</div>
<form action="login">
<input name="foo" value="bar"/>
<input name="username"/><input name="password"/>
<div>It is not unusual to see in an HTML page.</div>
</form></html>'''
et = ET.fromstring(html)
在Python 2或Python 3上运行它,您将看到此错误:
xml.etree.ElementTree.ParseError: undefined entity: line 7, column 38
那里有一些问答,比如这个和那个问答.他们暗示使用ElementTree.XMLParser().parser.UseForeignDTD(True)但我无法在Python 3.3和Python 3.4中使用它.
$ python3.3
Python 3.3.5 (v3.3.5:62cf4e77f785, Mar 9 2014, 01:12:57)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.etree.ElementTree as ET
>>> ET.XMLParser().parser
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'xml.etree.ElementTree.XMLParser' object has no attribute 'parser'
>>>
受这篇文章的启发,我们可以将一些XML定义添加到传入的原始HTML内容中,然后ElementTree可以开箱即用.
这适用于Python 2.6,2.7,3.3,3.4.
import xml.etree.ElementTree as ET
html = '''<html>
<div>Some reasonably well-formed HTML content.</div>
<form action="login">
<input name="foo" value="bar"/>
<input name="username"/><input name="password"/>
<div>It is not unusual to see in an HTML page.</div>
</form></html>'''
magic = '''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd" [
<!ENTITY nbsp ' '>
]>''' # You can define more entities here, if needed
et = ET.fromstring(magic + html)