Pandas read_html 返回原始 HTML 内容 [对于某些行/单元格/等]

Question

问题很简单，但我猜答案是“否”：

我有一个 HTML 表，我正在用 pandas.read_html 读取它，效果很好。但有些单元格（列）中包含图像、列表或其他显然被 read_html 丢弃的格式。我显然不希望 pandas 解析其中的任何内容，但是有什么方法可以让它返回原始 HTML，例如 DataFrame 单元格中的字符串，以便我可以自己解析它？

例子：

<table>
    <th>Column 1</th> 
    <th>Column 2</th>
    <tr>
        <td>Cell1</td> 
        <td>Cell2 <img src="http://www.link.com/image.jpg /></td>
    </tr>
    <tr>
        <td>Cell3</td> 
        <td>Cell4 <img src="http://www.website.com/picture.gif /></td>
    </tr>          
</table>

如果 Pandas 要解析这个，我可能只会从第 2 列中获取“Cell2”和“Cell4”。我想做的是以某种方式获取单元格的全部内容，包括标签的 [raw?] HTML 内容<img>。然后我可以自己解析它们。

Answer 1

没有任何选项可以pd.read_html实现您想要的功能。因此，当我尝试从加州国务院网站获取企业名称列表时，当 Pandas 自动解析 HTML 时，我得到了除名称之外的所有内容：

>>> bizname = 'poss'
>>> url = f'https://businesssearch.sos.ca.gov/CBS/SearchResults?filing=&SearchType=CORP&SearchCriteria={bizname}&SearchSubType=Begins'
>>> df = pd.read_html(url)[0]
>>> df
  Entity Number Registration Date         Status                                        Entity Name Jurisdiction      Agent for Service of Process
0      C2645412        04/02/2004         ACTIVE  View details for entity number 02645412  POSSU...      GEORGIA   ERESIDENTAGENT, INC. (C2702827)
1      C0786330        09/22/1976      DISSOLVED  View details for entity number 00786330  POSSU...   CALIFORNIA                        I. HALPERN
2      C2334141        03/01/2001  FTB SUSPENDED  View details for entity number 02334141  POSSU...   CALIFORNIA                   CLAIR G BURRILL
3      C0658630        11/08/1972  FTB SUSPENDED  View details for entity number 00658630  POSSU...   CALIFORNIA                               NaN
4      C1713121        09/23/1992  FTB SUSPENDED  View details for entity number 01713121  POSSU...   CALIFORNIA                LAWRENCE J. TURNER
5      C1207820        08/05/1983      DISSOLVED  View details for entity number 01207820  POSSU...   CALIFORNIA                          R L CARL
6      C3921531        06/27/2016         ACTIVE  View details for entity number 03921531  POSSU...   CALIFORNIA  REGISTERED AGENTS INC (C3365816)

该网站将公司名称隐藏在按钮后面。但您可以使用它requests来下载原始 html。然后，您可以用来bs4提取原始 HTML 表格以及您想要的任何特定行 ( <tr>) 或单元格 ( )。<td>

>>> soup = bs4.BeautifulSoup(requests.get(url).text)
>>> table = soup.find('table').findAll('tr')
>>> names = []
... for row in table:
...     names.append(getattr(row.find('button'), 'contents', [''])[0].strip())
>>> names
['',
 'POSSUM FILMS, INC',
 'POSSUM INC.',
 'POSSUM MEDIA, INC.',
 'POSSUM POINT PRODUCTIONS, INC.',
 'POSSUM PRODUCTIONS, INC.',
 'POSSUM-BILITY EXPRESS, INCORPORATED',
]
>>> df['Entity Name'] = names[1:]
>>> df['Entity Name'] = names[1:]
>>> df
  Entity Number Registration Date         Status                          Entity Name Jurisdiction      Agent for Service of Process
0      C2645412        04/02/2004         ACTIVE                    POSSUM FILMS, INC      GEORGIA   ERESIDENTAGENT, INC. (C2702827)
1      C0786330        09/22/1976      DISSOLVED                          POSSUM INC.   CALIFORNIA                        I. HALPERN
2      C2334141        03/01/2001  FTB SUSPENDED                   POSSUM MEDIA, INC.   CALIFORNIA                   CLAIR G BURRILL
3      C0658630        11/08/1972  FTB SUSPENDED       POSSUM POINT PRODUCTIONS, INC.   CALIFORNIA                               NaN
4      C1713121        09/23/1992  FTB SUSPENDED             POSSUM PRODUCTIONS, INC.   CALIFORNIA                LAWRENCE J. TURNER
5      C1207820        08/05/1983      DISSOLVED  POSSUM-BILITY EXPRESS, INCORPORATED   CALIFORNIA                          R L CARL
6      C3921531        06/27/2016         ACTIVE                      POSSUMS WELCOME   CALIFORNIA  REGISTERED AGENTS INC (C33658

这样做不能正确处理标题，因此如果需要，请不要忘记忽略第一行。