使用urllib2加载URL时如何设置TCP_NODELAY标志？

Question

我使用urllib2加载网页,我的代码是:

httpRequest = urllib2.Request("http:/www....com")
pageContent = urllib2.urlopen(httpRequest)
pageContent.readline()

如何获取要设置的套接字属性TCP_NODELAY？

在普通的套接字我会使用函数:

socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

Answer 1

如果需要在所使用的套接字上访问这种低级属性,则必须重载某些对象.

首先,您需要创建HTTPHandler的子类,在标准库中执行以下操作:

class HTTPHandler(AbstractHTTPHandler):

    def http_open(self, req):
        return self.do_open(httplib.HTTPConnection, req)

    http_request = AbstractHTTPHandler.do_request_

如您所见,它使用a HTTPConnection打开连接......您也必须覆盖它;)以升级connect()方法.

这样的事情应该是一个好的开始:

class LowLevelHTTPConnection(httplib.HTTPConnection):

    def connect(self):
        httplib.HTTPConnection.connect(self)
        self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)


class LowLevelHTTPHandler(HTTPHandler):

    def http_open(self, req):
        return self.do_open(LowLevelHTTPConnection, req)

urllib2足够智能,允许你子类化一些处理程序,然后使用它,urllib2.build_opener就是这样做的:

urllib2.install_opener(urllib2.build_opener(LowLevelHTTPHandler)) # tell urllib2 to use your HTTPHandler in replacement of the standard HTTPHandler
httpRequest = urllib2.Request("http:/www....com")
pageContent = urllib2.urlopen(httpRequest)
pageContent.readline()

Answer 2

对于请求,这些类似乎在request.packages.urllib3; 有两个类,HTTPConnection和HTTPSConnection.它们应该是模块顶层的monkeypatble:

from requests.packages.urllib3 import connectionpool

_HTTPConnection = connectionpool.HTTPConnection
_HTTPSConnection = connectionpool.HTTPSConnection

class HTTPConnection(_HTTPConnection):
    def connect(self):
        _HTTPConnection.connect(self)
        self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

class HTTPSConnection(_HTTPSConnection):
    def connect(self):
        _HTTPSConnection.connect(self)
        self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

connectionpool.HTTPConnection = HTTPConnection
connectionpool.HTTPSConnection = HTTPSConnection