urllib2 HTTP错误400:错误请求

Question

我有一段这样的代码

host = 'http://www.bing.com/search?q=%s&go=&qs=n&sk=&sc=8-13&first=%s' % (query, page)
req = urllib2.Request(host)
req.add_header('User-Agent', User_Agent)
response = urllib2.urlopen(req)

当我输入一个大于一个单词的查询,如"狗"时,我得到以下错误.

response = urllib2.urlopen(req)
File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 400, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 513, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 438, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 372, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 521, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request

任何人都可以指出我做错了什么？提前致谢.

Answer 1

"狗"返回400错误的原因是因为您没有转义URL的字符串.

如果你这样做:

import urllib, urllib2

quoted_query = urllib.quote(query)
host = 'http://www.bing.com/search?q=%s&go=&qs=n&sk=&sc=8-13&first=%s' % (quoted_query, page)
req = urllib2.Request(host)
req.add_header('User-Agent', User_Agent)
response = urllib2.urlopen(req)

它会工作.

但我强烈建议您使用请求而不是使用urllib/urllib2/httplib.它更容易,它将为您处理所有这些.

这与python请求的代码相同:

import requests

results = requests.get("http://www.bing.com/search", 
              params={'q': query, 'first': page}, 
              headers={'User-Agent': user_agent})

Answer 2

您需要urllib.quote()在'query'变量上使用:

query = urllib.quote(query)
host = 'http://www.bing.com/search?q=%s&go=&qs=n&sk=&sc=8-13&first=%s' % (query, page)

这会执行必要的URL转义以将空间转换big dog为big%20dog.