集成CherryPy Web框架,SQLAlchemy会话和lighttpd以提供高负载Web服务的最佳实践

Question

我正在开发一个支持lighttpd的CherryPy FastCGI服务器,其设置如下,以便在CherryPy控制器中使用ORM SQLAlchemy会话.然而,当我运行压力测试与约500回路14个并发请求,它开始给错误,如AttributeError: '_ThreadData' object has no attribute 'scoped_session_class'在open_dbsession()或AttributeError: 'Request' object has no attribute 'scoped_session_class'在close_dbsession()一段时间后.错误率总共约为50%.

只有当我在lighttpd后面运行服务器时才会发生这种情况,而不是直接运行时cherrypy.engine.start().确认connect()不会引发异常.

我也尝试将scoped_sessionto 的返回值赋予GlobalSession(就像在这里一样),但是它给出了错误UnboundExceptionError和其他SA级错误.(并发:10,循环:1000,错误率:16%.即使直接运行也会发生.)

有一些可能的原因,但我缺乏足够的知识来选择一个.
1. start_thread在FastCGI环境下订阅是否不可靠？它似乎open_dbsession()在之前被调用connect()
2.是否cherrypy.thread_data因某种原因被清除？

服务器代码

import sqlalchemy as sa  
from sqlalchemy.orm import session_maker, scoped_session

engine = sa.create_engine(dburi, strategy="threadlocal")  
GlobalSession = session_maker(bind=engine, transactional=False)

def connect(thread_index):  
    cherrypy.thread_data.scoped_session_class = scoped_session(GlobalSession)

def open_dbsession():  
    cherrypy.request.scoped_session_class = cherrypy.thread_data.scoped_session_class

def close_dbsession():  
    cherrypy.request.scoped_session_class.remove()


cherrypy.tools.dbsession_open = cherrypy.Tool('on_start_resource', open_dbsession)  
cherrypy.tools.dbsession_close = cherrypy.Tool('on_end_resource', close_dbsession)  
cherrypy.engine.subscribe('start_thread', connect)

lighttpd fastcgi config

...
var.server_name = "test"
var.server_root = "/path/to/root"
var.svc_env = "test"
fastcgi.server = (
  "/" => (
    "cherry.fcgi" => (
      "bin-path" => server_root + "/fcgi_" + server_name + ".fcgi",
      "bin-environment" => (
        "SVC_ENV" => svc_env
      ),
      "bin-copy-environment" => ("PATH", "LC_CTYPE"),
      "socket" => "/tmp/cherry_" + server_name + "." + svc_env + ".sock",
      "check-local" => "disable",
      "disable-time"    => 1,
      "min-procs"       => 1,
      "max-procs"       => 4,
    ),
  ),
)

编辑

• thread_index从原始源代码中恢复了代码示例中缺少的参数(感谢评论)
• 澄清错误不会立即发生
• 缩小了对lighttpd的条件

Answer 1

如果您查看plugins.ThreadManager.acquire_thread，您将看到该行self.bus.publish('start_thread', i)，其中i是所见线程的数组索引。订阅该start_thread频道的任何侦听器都需要接受该i值作为位置参数。因此，重写您的连接函数：def connect(i):

我猜这会以某种方式悄然失败；我会看看是否可以找到它、测试并修复它。