标签: html5lib

在beautifulsoup的上下文中lxml和html5lib之间的区别

在beautifulsoup的上下文中,lxml和html5lib解析器的功能有区别吗?我正在尝试学习使用BS4并使用以下代码构造 -

ret = requests.get('http://www.olivegarden.com')
soup = BeautifulSoup(ret.text, 'html5lib')
for item in soup.find_all('a'): 
    print item['href']
Run Code Online (Sandbox Code Playgroud)

我开始使用lxml作为解析器,但注意到对于某些网站,即使页面中有有效链接,也永远不会输入for循环.同一页面适用于html5ib解析器.是否有任何特定类型的页面可能无法与lxml一起使用?

我在Ubuntu上使用python-lxml 2.3.2-1和libxml2 2.7.8.dfsg-5.1ubunt和html5lib-1.0b3

编辑:我更新到lxml 3.1.2仍然看到相同的问题.在运行3.0.x的mac上,正在正确解析同一页面.有问题的网站是www.olivegarden.com

python lxml beautifulsoup html5lib

5
推荐指数
1
解决办法
3136
查看次数

如何修复html5lib中的"意外关键字参数'useChardet'"

我正在使用html5lib并在将其更新到最新版本后,我不断收到此错误:

Traceback (most recent call last):
  File "/home/travis/build/freelawproject/juriscraper/tests/test_everything.py", line 119, in test_scrape_all_example_files
    site.parse()
  File "/home/travis/build/freelawproject/juriscraper/juriscraper/AbstractSite.py", line 95, in parse
    self.html = self._download()
  File "/home/travis/build/freelawproject/juriscraper/juriscraper/AbstractSite.py", line 384, in _download
    html_tree = self._make_html_tree(text)
  File "/home/travis/build/freelawproject/juriscraper/juriscraper/opinions/united_states/federal_appellate/ca11_u.py", line 26, in _make_html_tree
    e = html5parser.document_fromstring(text)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/lxml/html/html5parser.py", line 64, in document_fromstring
    return parser.parse(html, useChardet=guess_charset).getroot()
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/html5parser.py", line 235, in parse
    self._parse(stream, False, None, *args, **kwargs)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/html5parser.py", line 85, in _parse
    self.tokenizer = _tokenizer.HTMLTokenizer(stream, parser=self, **kwargs)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/_tokenizer.py", line 36, in __init__
    self.stream = HTMLInputStream(stream, …
Run Code Online (Sandbox Code Playgroud)

python html5lib

5
推荐指数
1
解决办法
362
查看次数

BeautifulSoup find_all限制为50个结果?

我正在尝试使用BeautifulSoup从页面获取结果:

req_url = 'http://www.xscores.com/soccer/livescores/25-02'
request = requests.get(req_url)
content = request.content
soup = BeautifulSoup(content, "html.parser")
scores = soup.find_all('tr', {'style': 'height:18px;'}, limit=None)
print(len(scores))
>50
Run Code Online (Sandbox Code Playgroud)

我读了之前的解决方案:美丽的汤findAll找不到所有 ,我尝试了html.parser,lxml和html5lib,但没有一个返回超过50个结果.有什么建议?

谢谢

python lxml beautifulsoup html5lib

5
推荐指数
1
解决办法
1261
查看次数

html5lib已安装但BeautifulSoup找不到它

我已经安装了 html5lib 包。我确定,因为当我尝试安装它时,我收到一条消息,表明它已安装。

pip install html5lib
Requirement already satisfied: html5lib in ./anaconda/lib/python3.5/site-packages
Run Code Online (Sandbox Code Playgroud)

我也能够成功导入该包。

import html5lib
Run Code Online (Sandbox Code Playgroud)

然而,当我尝试使用解析器作为 BeautifulSoup 构造函数的一部分时

soup = BeautifulSoup(response.data, 'html5lib')
Run Code Online (Sandbox Code Playgroud)

我不成功

FeatureNotFound: Couldn't find a tree builder with the features you requested: html5lib. Do you need to install a parser library?
Run Code Online (Sandbox Code Playgroud)

是否可以强制 bs4 查找已安装的解析器并成功找到 html5lib?

python beautifulsoup html5lib

5
推荐指数
1
解决办法
8556
查看次数

PHP 中带有 html5lib 的 Xpath

我有这个不起作用的基本代码。如何在 html5lib php 中使用 Xpath?或以任何其他方式使用 HTML5 的 Xpath。

$url = 'http://en.wikipedia.org/wiki/PHP';
$response = GuzzleHttp\get($url);

$html5 = new Masterminds\HTML5();
$dom = $html5->loadHTML($response);

$xpath = new DOMXPath($dom);

$elements = $xpath->query('//h1');
//$elements = $dom->getElementsByTagName('h1');

foreach ($elements as $element)
{
    var_dump($element);
}
Run Code Online (Sandbox Code Playgroud)

未找到任何元素。使用$xpath->query('.')作品来获取根元素(通常 xpath 似乎有效)。$dom->getElementsByTagName('h1')正在工作。

html php xpath html5lib

4
推荐指数
1
解决办法
981
查看次数

html5lib:TypeError:__ init __()得到了一个意外的关键字参数'encoding'

我正在尝试安装html5lib.起初我试图安装最新版本(8或9个9),但它与我的BeautifulSoup发生冲突,所以我决定尝试更老的版本(0.9999999,7 个9).我安装了它,但是当我尝试使用它时:

>>> with urlopen("http://example.com/") as f:
    document = html5lib.parse(f, encoding=f.info().get_content_charset())
Run Code Online (Sandbox Code Playgroud)

我收到一个错误:

Traceback (most recent call last):
  File "<pyshell#11>", line 2, in <module>
    document = html5lib.parse(f, encoding=f.info().get_content_charset())
  File "C:\Python\Python35-32\lib\site-packages\html5lib\html5parser.py", line 35, in parse
    return p.parse(doc, **kwargs)
  File "C:\Python\Python35-32\lib\site-packages\html5lib\html5parser.py", line 235, in parse
    self._parse(stream, False, None, *args, **kwargs)
  File "C:\Python\Python35-32\lib\site-packages\html5lib\html5parser.py", line 85, in _parse
    self.tokenizer = _tokenizer.HTMLTokenizer(stream, parser=self, **kwargs)
  File "C:\Python\Python35-32\lib\site-packages\html5lib\_tokenizer.py", line 36, in __init__
    self.stream = HTMLInputStream(stream, **kwargs)
  File "C:\Python\Python35-32\lib\site-packages\html5lib\_inputstream.py", line 151, in HTMLInputStream
    return HTMLBinaryInputStream(source, **kwargs) …
Run Code Online (Sandbox Code Playgroud)

beautifulsoup html5lib web-scraping

4
推荐指数
1
解决办法
4749
查看次数

ImportError:html5lib中没有名为base的模块

我突然无法启动Django服务器了,运行检查:

python manage.py check
Run Code Online (Sandbox Code Playgroud)

显示以下错误:

    apps.populate(settings.INSTALLED_APPS)
  File "/home/aym/.virtualenvs/prod_env/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/home/aym/.virtualenvs/prod_env/local/lib/python2.7/site-packages/django/apps/config.py", line 198, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/aym/workspace/CompanyP/apps/account/models.py", line 18, in <module>
    from apps.groups.models import SubscriptionGroup
  File "/home/aym/workspace/CompanyP/apps/subscription/models.py", line 8, in <module>
    from apps.member.models import Member
  File "/home/aym/.virtualenvs/prod_env/local/lib/python2.7/site-packages/bleach/__init__.py", line 5, in <module>
    from bleach.linkifier import 
  File "/home/aym/.virtualenvs/prod_env/local/lib/python2.7/site-packages/bleach/linkifier.py", line 5, in <module>
    from html5lib.filters.base import Filter
ImportError: No module named base
Run Code Online (Sandbox Code Playgroud)

你知道吗?

python django importerror html5lib requirements.txt

4
推荐指数
1
解决办法
4483
查看次数

使用pip安装期间传输_encoding错误

在尝试通过pip安装任何python包时,我在parse()中得到了意外的arg:keyword编码.

我遇到了这个问题,因为我为python 3.6安装了tensorflow,这可能导致了html5lib和setuptools的一些问题.使用tar.gz文件(管理员安装)重新安装了html5lib1.0b10,但问题仍然存在.请帮忙!!

pip install spacy
Collecting spacy
Exception:
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pip\commands\install.py", line 335, in run
    wb.build(autobuilding=True)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pip\wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pip\req\req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "C:\ProgramData\Anaconda3\lib\site-packages\pip\req\req_set.py", line 554, in _prepare_file
    require_hashes
  File "C:\ProgramData\Anaconda3\lib\site-packages\pip\req\req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pip\index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pip\index.py", line 423, in find_all_candidates
    for page in …
Run Code Online (Sandbox Code Playgroud)

pip setuptools html5lib python-3.x

4
推荐指数
2
解决办法
5791
查看次数

使用html5lib或bleach删除<style> ... </ style>标记的内容

我一直在使用优秀的漂白库来删除错误的HTML.

我有一堆HTML文档已经从Microsoft Word粘贴,包含以下内容:

<STYLE> st1:*{behavior:url(#ieooui) } </STYLE>
Run Code Online (Sandbox Code Playgroud)

使用漂白(style隐藏不允许标记),让我:

st1:*{behavior:url(#ieooui) }
Run Code Online (Sandbox Code Playgroud)

哪个没用.Bleach似乎只有以下选项:

  • 逃生标签;
  • 删除标签(但不删除其内容).

我正在寻找第三种选择 - 删除标签及其内容.

有没有办法使用漂白或html5lib来完全删除style标签及其内容?html5lib的文档并没有太大帮助.

python django html5lib

3
推荐指数
1
解决办法
1103
查看次数

python3-没有名为“ html5lib”的模块

我正在运行需要的python3程序,html5lib但收到错误消息No module named 'html5lib'。

这是终端的两个会话:

sam@pc ~ $ python
Python 2.7.9 (default, Mar  1 2015, 12:57:24) 
[GCC 4.9.2] on linux2
>>> import html5lib
>>> html5lib.__file__
'/usr/local/lib/python2.7/dist-packages/html5lib/__init__.pyc'
>>> quit()

sam@pc ~ $ python3
Python 3.4.2 (default, Oct  8 2014, 10:45:20) 
[GCC 4.9.1] on linux
>>> import html5lib
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'html5lib'
>>> 
Run Code Online (Sandbox Code Playgroud)

问题出在哪里?

html5lib python-3.x

3
推荐指数
1
解决办法
5000
查看次数