如何使用urllib2.urlopen上默认用户代理以外的用户代理下载网页?
我无法使用urllib2打开一个特定的URL.同样的方法适用于其他网站,例如"http://www.google.com",但不适用于此网站(在浏览器中也可以正常显示).
我的简单代码:
from BeautifulSoup import BeautifulSoup
import urllib2
url="http://www.experts.scival.com/einstein/"
response=urllib2.urlopen(url)
html=response.read()
soup=BeautifulSoup(html)
print soup
Run Code Online (Sandbox Code Playgroud)
任何人都可以帮助我使它工作吗?
这是我得到的错误:
Traceback (most recent call last):
File "/Users/jontaotao/Documents/workspace/MedicalSchoolInfo/src/AlbertEinsteinCollegeOfMedicine_SciValExperts/getlink.py", line 12, in <module>
response=urllib2.urlopen(url);
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 400, in open
response = meth(req, response)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 513, in http_response
'http', request, response, code, msg, hdrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 432, in error
result = self._call_chain(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 372, in _call_chain
result = func(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", …Run Code Online (Sandbox Code Playgroud)