小编sha*_*iff的帖子

如何使用美丽的汤从URL中提取内容时设置会话cookie?

考虑一下代码:

from bs4 import BeautifulSoup
from urllib.request import urlopen
content = urlopen('https://example.net/users/101')
soup = BeautifulSoup(content)
divTag = soup.find_all("div", {"class":"classname"})
print(divTag)
for tag in divTag:
   ulTags = tag.find_all("ul", {"class":"classname"})
   for tag in ulTags:
       aTags = tag.find_all("li")
       for tag in aTags:
           name = tag.find('a')['href']
           print(name)
Run Code Online (Sandbox Code Playgroud)

如果我使用,

content = open("try.html","r")
Run Code Online (Sandbox Code Playgroud)

我得到了所需的输出.

在这里,只有在输入用户名和密码后才能访问example.net.虽然解析是正确完成的,但上面的代码不会打印任何内容.如何将会话cookie值添加到此代码中?

python cookies session beautifulsoup web-scraping

6
推荐指数
1
解决办法
6887
查看次数

标签 统计

beautifulsoup ×1

cookies ×1

python ×1

session ×1

web-scraping ×1