Python 2.7:流式HTTP服务器支持一个端口上的多个连接

Question

我正在寻找一个标准的Python 2.7软件包,它提供了一个HTTP服务器,可以在同一个端口号上同时进行流连接.

嘿你们那里的主持人,请停止将我的问题标记为想要以非流方式提供的问题的副本,例如: python中的多线程Web服务器.不,我不想要一个ThreadingMixIn只是收集响应并将其作为一个单元返回的黑客.

换句话说,我正在寻找标准方法来执行以下示例程序所做的事情 - 但是没有自己编写整个HTTP服务器.

import time, socket, threading

sock = socket.socket (socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostname()
port = 8000

sock.bind((host, port))
sock.listen(1)

# my OWN HTTP server... Oh man, this is bad style.
HTTP = "HTTP/1.1 200 OK\nContent-Type: text/html; charset=UTF-8\n\n"

class Listener(threading.Thread):

    def __init__(self):
        threading.Thread.__init__(self)
        self.daemon = True # stop Python from biting ctrl-C
        self.start()

    def run(self):
        conn, addr = sock.accept()
        conn.send(HTTP)

        # serve up an infinite stream
        i = 0
        while True:
            conn.send("%i " % i)
            time.sleep(0.1)
            i += 1

[Listener() for i in range(100)]
time.sleep(9e9)

所以首先我试过:

# run with this command:
#    gunicorn -k gevent myapp:app
import time

def app(environ, start_response):
    data = b"Hello, World!\n"
    start_response("200 OK", [
        ("Content-Type", "text/plain"),
        ("Content-Length", str(len(data)))
    ])
    for i in range(5):
        time.sleep(1)
        yield "Hello %i\n" % i

# https://stackoverflow.com/questions/22739394/streaming-with-gunicorn

但不幸的是,即使有了,也没有流-k gevent.

更新:似乎gunicorn正在尝试使用keepalive,这需要使用Chunked Transfer Coding last-chunk.一个快速的资料显示,它没有实现这一点.所以我可能需要一个更漂亮的HTTP服务器,或者更简单的一个(就像我上面的第一个例子,基于socket),它不会干扰keepalive(对于大流来说这是非常愚蠢的).

那么我试过:

import time
import threading

import BaseHTTPServer

class Handler(BaseHTTPServer.BaseHTTPRequestHandler):

    def do_GET(self):
        if self.path != '/':
            self.send_error(404, "Object not found")
            return
        self.send_response(200)
        self.send_header('Content-type', 'text/html; charset=utf-8')
        self.end_headers()

        # serve up an infinite stream
        i = 0
        while True:
            self.wfile.write("%i " % i)
            time.sleep(0.1)
            i += 1

class Listener(threading.Thread):

    def __init__(self, i):
        threading.Thread.__init__(self)
        self.i = i
        self.daemon = True
        self.start()

    def run(self):
        server_address = ('', 8000+self.i) # How to attach all of them to 8000?
        httpd = BaseHTTPServer.HTTPServer(server_address, Handler)
        httpd.serve_forever()

[Listener(i) for i in range(100)]
time.sleep(9e9)

这是相当不错的,但我必须分配100个端口号有点烦人.这将需要一个讨厌的客户端重定向来使浏览器到达下一个可用端口(好吧,好吧,我可以用JavaScript隐藏它,但它不是那么优雅.我宁愿编写自己的HTTP服务器而不是那样做).

必须有一种干净的方法来BaseHTTPServer在一个端口上获取所有侦听器,因为它是设置Web服务器的标准方法.或者可能gunicorn或者某些包装可以可靠地流动？

Answer 1

默认BaseHTTPServer设置会在每个侦听器上重新绑定一个新套接字,如果所有侦听器都在同一个端口上,则无法在Linux中运行.在BaseHTTPServer.HTTPServer()呼叫和serve_forever()呼叫之间更改这些设置.

以下示例在同一端口上启动100个处理程序线程,每个处理程序都通过BaseHTTPServer.

import time, threading, socket, SocketServer, BaseHTTPServer

class Handler(BaseHTTPServer.BaseHTTPRequestHandler):

    def do_GET(self):
        if self.path != '/':
            self.send_error(404, "Object not found")
            return
        self.send_response(200)
        self.send_header('Content-type', 'text/html; charset=utf-8')
        self.end_headers()

        # serve up an infinite stream
        i = 0
        while True:
            self.wfile.write("%i " % i)
            time.sleep(0.1)
            i += 1

# Create ONE socket.
addr = ('', 8000)
sock = socket.socket (socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(addr)
sock.listen(5)

# Launch 100 listener threads.
class Thread(threading.Thread):
    def __init__(self, i):
        threading.Thread.__init__(self)
        self.i = i
        self.daemon = True
        self.start()
    def run(self):
        httpd = BaseHTTPServer.HTTPServer(addr, Handler, False)

        # Prevent the HTTP server from re-binding every handler.
        # https://stackoverflow.com/questions/46210672/
        httpd.socket = sock
        httpd.server_bind = self.server_close = lambda self: None

        httpd.serve_forever()
[Thread(i) for i in range(100)]
time.sleep(9e9)