使用urllib2代理

Question

我打开网址:

site = urllib2.urlopen('http://google.com')

我想要做的是用同样的方式连接我在某个地方告诉我:

site = urllib2.urlopen('http://google.com', proxies={'http':'127.0.0.1'})

但那也不起作用.

我知道urllib2有类似代理处理程序的东西,但我不记得那个功能.

Answer 1

proxy = urllib2.ProxyHandler({'http': '127.0.0.1'})
opener = urllib2.build_opener(proxy)
urllib2.install_opener(opener)
urllib2.urlopen('http://www.google.com')

Answer 2

您必须安装ProxyHandler

urllib2.install_opener(
    urllib2.build_opener(
        urllib2.ProxyHandler({'http': '127.0.0.1'})
    )
)
urllib2.urlopen('http://www.google.com')

Answer 3

您可以使用环境变量设置代理.

import os
os.environ['http_proxy'] = '127.0.0.1'
os.environ['https_proxy'] = '127.0.0.1'

urllib2将以这种方式自动添加代理处理程序.您需要单独为不同的协议设置代理,否则它们将失败(就不通过代理而言),请参阅下文.

例如:

proxy = urllib2.ProxyHandler({'http': '127.0.0.1'})
opener = urllib2.build_opener(proxy)
urllib2.install_opener(opener)
urllib2.urlopen('http://www.google.com')
# next line will fail (will not go through the proxy) (https)
urllib2.urlopen('https://www.google.com')

代替

proxy = urllib2.ProxyHandler({
    'http': '127.0.0.1',
    'https': '127.0.0.1'
})
opener = urllib2.build_opener(proxy)
urllib2.install_opener(opener)
# this way both http and https requests go through the proxy
urllib2.urlopen('http://www.google.com')
urllib2.urlopen('https://www.google.com')

Answer 4

要使用默认系统代理(例如,来自http_support环境变量),以下内容适用于当前请求(无需将其全局安装到urllib2中):

url = 'http://www.example.com/'
proxy = urllib2.ProxyHandler()
opener = urllib2.build_opener(proxy)
in_ = opener.open(url)
in_.read()