相关疑难解决方法(0)

在urllib2.urlopen上更改用户代理

如何使用urllib2.urlopen上默认用户代理以外的用户代理下载网页?

python user-agent urllib2

97
推荐指数
7
解决办法
14万
查看次数

对于在浏览器中显示正常的网站,urllib2返回404

我无法使用urllib2打开一个特定的URL.同样的方法适用于其他网站,例如"http://www.google.com",但不适用于此网站(在浏览器中也可以正常显示).

我的简单代码:

from BeautifulSoup import BeautifulSoup
import urllib2

url="http://www.experts.scival.com/einstein/"
response=urllib2.urlopen(url)
html=response.read()
soup=BeautifulSoup(html)
print soup
Run Code Online (Sandbox Code Playgroud)

任何人都可以帮助我使它工作吗?

这是我得到的错误:

Traceback (most recent call last):
  File "/Users/jontaotao/Documents/workspace/MedicalSchoolInfo/src/AlbertEinsteinCollegeOfMedicine_SciValExperts/getlink.py", line 12, in <module>
    response=urllib2.urlopen(url);
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 400, in open
    response = meth(req, response)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 432, in error
    result = self._call_chain(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", …
Run Code Online (Sandbox Code Playgroud)

html python url urllib2

9
推荐指数
1
解决办法
2万
查看次数

标签 统计

python ×2

urllib2 ×2

html ×1

url ×1

user-agent ×1