如何从html页面中提取文本？

Question

例如,网页是链接:

https://www.architecture.com/FindAnArchitect/FAAPractices.aspx?display=50

我必须有公司的名称及其地址和网站.我尝试了以下将html转换为文本:

import nltk   
from urllib import urlopen

url = "https://www.architecture.com/FindAnArchitect/FAAPractices.aspx display=50"    
html = urlopen(url).read()    
raw = nltk.clean_html(html)  
print(raw)

但它返回错误:

ImportError: cannot import name 'urlopen

Answer 1

彼得伍德回答了你的问题(链接).

import urllib.request

uf = urllib.request.urlopen(url)
html = uf.read()

但是,如果要提取数据(例如公司名称,地址和网站),则需要获取HTML源并使用HTML解析器对其进行解析.

我建议requests用于获取HTML源代码并BeautifulSoup解析生成的HTML并提取所需的文本.

这是一个小狙击手,将为您提供一个良好的开端.

import requests
from bs4 import BeautifulSoup

link = "https://www.architecture.com/FindAnArchitect/FAAPractices.aspx?display=50"

html = requests.get(link).text

"""If you do not want to use requests then you can use the following code below 
   with urllib (the snippet above). It should not cause any issue."""
soup = BeautifulSoup(html, "lxml")
res = soup.findAll("article", {"class": "listingItem"})
for r in res:
    print("Company Name: " + r.find('a').text)
    print("Address: " + r.find("div", {'class': 'address'}).text)
    print("Website: " + r.find_all("div", {'class': 'pageMeta-item'})[3].text)