通过HTTPS使用gevent/grequests进行奇怪的阻止行为

Question

以下代码每200ms发送一次请求,并且应该在它们到来时异步处理响应.

通过HTTP,它按预期工作 - 每200ms发送一个请求,并在响应到达时独立调用响应回调.但是,通过HTTPS,只要响应到达,请求就会显着延迟(即使我的响应处理程序不起作用).对于每个请求,响应回调似乎被调用两次,一次是零长度响应(编辑:这是因为重定向并且似乎与阻塞问题无关,感谢Padraic).

什么可能导致这种阻止行为通过HTTPS？(www.bbc.co.uk这只是一个地理位置远离我的例子,但它发生在我测试过的所有服务器上).

grequests_test.py

import time
import sys
import grequests
import gevent

def cb(res, **kwargs):
    print("**** Response", time.time(), len(res.text))

for i in range(10):
    unsent = grequests.get(sys.argv[1], hooks={'response': cb})
    print("Request", time.time())
    grequests.send(unsent, grequests.Pool(1))
    gevent.sleep(0.2)
gevent.sleep(5)

$ ipython2 grequests_test.py 'http://www.bbc.co.uk' (预期结果)

('Request', 1459050191.499266)
('Request', 1459050191.701466)
('Request', 1459050191.903223)
('Request', 1459050192.10403)
('Request', 1459050192.305626)
('**** Response', 1459050192.099185, 179643)
('Request', 1459050192.506476)
('**** Response', 1459050192.307869, 179643)
('Request', 1459050192.707745)
('**** Response', 1459050192.484711, 179643)
('Request', 1459050192.909376)
('**** Response', 1459050192.696583, 179643)
('Request', 1459050193.110528)
('**** Response', 1459050192.870476, 179643)
('Request', 1459050193.311601)
('**** Response', 1459050193.071679, 179639)
('**** Response', 1459050193.313615, 179680)
('**** Response', 1459050193.4959, 179643)
('**** Response', 1459050193.687054, 179680)
('**** Response', 1459050193.902827, 179639)

ipython2 grequests_test.py 'https://www.bbc.co.uk' (请求发送迟到)

('Request', 1459050203.24336)
('Request', 1459050203.44473)
('**** Response', 1459050204.423302, 0)
('Request', 1459050204.424748) <------------- THIS REQUEST TIME IS LATE
('**** Response', 1459050205.294426, 0)
('Request', 1459050205.296722)
('Request', 1459050205.497924)
('**** Response', 1459050206.456572, 0)
('Request', 1459050206.457875)
('**** Response', 1459050207.363188, 0)
('**** Response', 1459050208.247189, 0)
('Request', 1459050208.249579)
('**** Response', 1459050208.250645, 179643)
('**** Response', 1459050208.253638, 179643)
('Request', 1459050208.451083)
('**** Response', 1459050209.426556, 0)
('Request', 1459050209.428032)
('**** Response', 1459050209.428929, 179643)
('**** Response', 1459050210.331425, 0)
('**** Response', 1459050211.247793, 0)
('Request', 1459050211.251574)
('**** Response', 1459050211.252321, 179643)
('**** Response', 1459050211.25519, 179680)
('**** Response', 1459050212.397186, 0)
('**** Response', 1459050213.299109, 0)
('**** Response', 1459050213.588854, 179588)
('**** Response', 1459050213.590434, 179643)
('**** Response', 1459050213.593731, 179643)
('**** Response', 1459050213.90507, 179643)
('**** Response', 1459050213.909386, 179643)

请注意,第一个响应似乎在下一个请求应该已发送但很长时间后才会到达.在第一个响应到来之前,为什么睡眠没有返回,下一个请求被发送？

Answer 1

请求的当前迭代包含以下内容：

from gevent import monkey as curious_george
curious_george.patch_all(thread=False, select=False)

有问题的部分是select=False- 删除它或手动调用monkey.patch_select()可以解决问题。我不确定这是否还有其他副作用。