使用gevent.queue.Queue.get():gevent.hub.LoopExit:'此操作将永远阻止'

Question

过去几天我一直在尝试将事件流整合到我的烧瓶应用程序中,并在我的本地测试中取得了良好的效果,但在我的服务器上使用uWSGI运行应用程序时更糟糕一些.我的代码基本上建立在flask 的例子之上.我正在使用python 3.4.2.

问题

在我的uWSGI服务器上运行应用程序时,gevent.hub.LoopExit: 'This operation would block forever'.只要客户端尝试连接到/streaming端点,就会引发该应用程序.我的假设是这是由get()无限期地调用空队列引起的.

完全追溯:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/werkzeug/wsgi.py", line 691, in __next__
    return self._next()
  File "/usr/lib/python3/dist-packages/werkzeug/wrappers.py", line 81, in _iter_encoded
    for item in iterable:
  File "./voting/__init__.py", line 49, in gen
    result = queue.get(block=True)
  File "/usr/local/lib/python3.4/dist-packages/gevent/queue.py", line 284, in get
    return self.__get_or_peek(self._get, block, timeout)
  File "/usr/local/lib/python3.4/dist-packages/gevent/queue.py", line 261, in __get_or_peek
    result = waiter.get()
  File "/usr/local/lib/python3.4/dist-packages/gevent/hub.py", line 878, in get
    return self.hub.switch()
  File "/usr/local/lib/python3.4/dist-packages/gevent/hub.py", line 609, in switch
    return greenlet.switch(self)
gevent.hub.LoopExit: ('This operation would block forever', <Hub at 0x7f717f40f5a0 epoll default pending=0 ref=0 fileno=6>)

我的代码

该/streaming端点:

@app.route("/streaming", methods=["GET", "OPTIONS"])
def streaming():
    def gen():
        queue = Queue()
        subscriptions.add_subscription(session_id, queue)

        try:
            while True:
                result = queue.get()      # Where the Exception is raised
                ev = ServerSentEvent(json.dumps(result["data"]), result["type"])
                yield ev.encode()

        except GeneratorExit:  # TODO Need a better method to detect disconnecting
            subscriptions.remove_subscription(session_id, queue)

    return Response(gen(), mimetype="text/event-stream")

将事件添加到队列:

def notify():
    msg = {"type": "users", "data": db_get_all_registered(session_id)}
    subscriptions.add_item(session_id, msg)      # Adds the item to the relevant queues.

gevent.spawn(notify)

如前所述,它在本地运行良好werkzeug:

from app import app
from gevent.wsgi import WSGIServer
from werkzeug.debug import DebuggedApplication

a = DebuggedApplication(app, evalex=True)
server = WSGIServer(("", 5000), a)
server.serve_forever()

我试过的

• 猴子修补monkey.patch_all().

• 切换Queue到JoinableQueue.

• gevent.sleep(0)与...结合Queue.get().

Answer 1

该异常基本上意味着在该循环/线程中没有其他greenlet切换到.因此,当greenlet进入阻塞(queue.get())时,集线器无处可去,没有其他事可做.

相同的代码可以在gevent的WSGIServer中运行,因为服务器本身是一个运行socket.accept循环的greenlet,因此总会有另一个greenlet切换到.但显然uwsgi并不是那样的.

解决这个问题的方法是安排其他运行的greenlets.例如,不是生成greenlet以按需通知,而是安排这样的greenlet已经在其自己的队列上运行和阻塞.