Loi*_*icM 8 python beautifulsoup python-3.x bs4
我今天在使用BeautifulSoup时遇到了一种非常奇怪的行为.
我们来看一个非常简单的html片段:
<html><body><ix:nonfraction>lele</ix:nonfraction></body></html>
我试图<ix:nonfraction>用BeautifulSoup 获取标签的内容.
使用该find方法时一切正常:
from bs4 import BeautifulSoup
html = "<html><body><ix:nonfraction>lele</ix:nonfraction></body></html>"
soup = BeautifulSoup(html, 'lxml') # The parser used here does not matter
soup.find('ix:nonfraction')
>>> <ix:nonfraction>lele</ix:nonfraction>
但是,在尝试使用该find_all方法时,我希望返回一个包含此单个元素的列表,但事实并非如此!
soup.find_all('ix:nonfraction')
>>> []
事实上,find_all每当我正在搜索的标签中出现冒号时,似乎都会返回一个空列表.
我已经能够在两台不同的计算机上重现这个问题.
有没有人有解释,更重要的是,有一个解决方法？我需要使用该find_all方法只是因为我的实际案例要求我在整个html页面上获取所有这些标签.
@ yosemite_k的解决方案工作的原因是因为在bs4的源代码中,它正在跳过导致此行为的特定条件.事实上,你可以做很多变化,这会产生同样的结果.例子:
soup.find_all({"ix:nonfraction"})
soup.find_all('ix:nonfraction', limit=1)
soup.find_all('ix:nonfraction', text=True)
下面是来自beautifulsoup源代码的片段,显示了当你打电话find或发生时会发生什么find_all.你会看到,find只是调用find_all带limit=1.在_find_all其中检查条件:
if text is None and not limit and not attrs and not kwargs:
如果它达到了那个条件,那么它最终可能会达到这个条件:
# Optimization to find all tags with a given name.
if name.count(':') == 1:
如果它在那里,那么它重新分配name:
# This is a name with a prefix.
prefix, name = name.split(':', 1)
这是你的行为不同的地方.只要find_all不符合任何先前条件,那么你就会找到元素.
beautifulsoup4 == 4.6.0
def find(self, name=None, attrs={}, recursive=True, text=None,
**kwargs):
"""Return only the first child of this Tag matching the given
criteria."""
r = None
l = self.find_all(name, attrs, recursive, text, 1, **kwargs)
if l:
r = l[0]
return r
findChild = find
def find_all(self, name=None, attrs={}, recursive=True, text=None,
limit=None, **kwargs):
"""Extracts a list of Tag objects that match the given
criteria. You can specify the name of the Tag and any
attributes you want the Tag to have.
The value of a key-value pair in the 'attrs' map can be a
string, a list of strings, a regular expression object, or a
callable that takes a string and returns whether or not the
string matches for some custom definition of 'matches'. The
same is true of the tag name."""
generator = self.descendants
if not recursive:
generator = self.children
return self._find_all(name, attrs, text, limit, generator, **kwargs)
def _find_all(self, name, attrs, text, limit, generator, **kwargs):
"Iterates over a generator looking for things that match."
if text is None and 'string' in kwargs:
text = kwargs['string']
del kwargs['string']
if isinstance(name, SoupStrainer):
strainer = name
else:
strainer = SoupStrainer(name, attrs, text, **kwargs)
if text is None and not limit and not attrs and not kwargs:
if name is True or name is None:
# Optimization to find all tags.
result = (element for element in generator
if isinstance(element, Tag))
return ResultSet(strainer, result)
elif isinstance(name, str):
# Optimization to find all tags with a given name.
if name.count(':') == 1:
# This is a name with a prefix.
prefix, name = name.split(':', 1)
else:
prefix = None
result = (element for element in generator
if isinstance(element, Tag)
and element.name == name
and (prefix is None or element.prefix == prefix)
)
return ResultSet(strainer, result)
results = ResultSet(strainer)
while True:
try:
i = next(generator)
except StopIteration:
break
if i:
found = strainer.search(i)
if found:
results.append(found)
if limit and len(results) >= limit:
break
return results
| 归档时间： |
|
| 查看次数： |
2792 次 |
| 最近记录： |