小编Var*_*run的帖子

使用 python 抓取 .aspx 页面

我是网络抓取游戏的新手。我正在尝试废弃以下网站: http://www.foodemissions.com/foodemissions/Calculator.aspx

使用在 Internet 上找到的资源,我整理了以下 HTTP POST 请求:

import urllib
from bs4 import BeautifulSoup

headers = {
    'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko)  Chrome/24.0.1312.57 Safari/537.17',
    'Content-Type': 'application/x-www-form-urlencoded',
    'Accept-Encoding': 'gzip,deflate,sdch',
    'Accept-Language': 'en-US,en;q=0.8',
    'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
}

class MyOpener(urllib.FancyURLopener):
    version = 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17'

myopener = MyOpener()
url = 'http://www.foodemissions.com/foodemissions/Calculator.aspx'
# first HTTP request without form data
f = myopener.open(url)
soup_dummy = BeautifulSoup(f,"html5lib")
# parse and retrieve two vital form values
viewstate …
Run Code Online (Sandbox Code Playgroud)

python asp.net beautifulsoup

8
推荐指数
1
解决办法
2万
查看次数

标签 统计

asp.net ×1

beautifulsoup ×1

python ×1