强制转换为Unicode:需要字符串或缓冲区,找到Tag

Question

我正在尝试进行网页抓取并使用以下代码:

import mechanize
from bs4 import BeautifulSoup

url = "http://www.indianexpress.com/news/indian-actions-discriminating-against-us-exp/1131015/"
br =  mechanize.Browser()
htmltext = br.open(url).read()
articletext = ""
soup = BeautifulSoup(htmltext)
for tag in soup.findAll('p'):
    articletext += tag.contents[0]
print articletext

但我收到以下错误:

Traceback (most recent call last):
  File "C:/Python27/crawler/express.py", line 15, in <module>
    articletext += tag.contents[0]
TypeError: coercing to Unicode: need string or buffer, Tag found

有人可以帮我解决这个错误,我是Python编程的新手.

Answer 1

对于<p>找到的至少一个标签,tag.contents[0]是一个Tag对象,而不是文本.对于您找到的特定网址,它是一个<hr>标记:

>>> tag
<p><hr> </hr></p>

为什么不用tag.text而不是tag.contents[0]？

for tag in soup.findAll('p'):
    articletext += tag.text