运行多个龙卷风进程

Question

我已经阅读了有关如何运行N个Tornado进程的各种文章和教程,其中N =核心数.我的代码正在运行,在所有16个核心上运行,但我设法搞砸了它,我需要新的眼睛.

import tornado.ioloop
import tornado.web
import tornado.httpserver

from core import settings
from core import coreService
import services

from tornado.options import define, options, parse_command_line

define("port", default=settings.SERVER_PORT, help="run on the given port", type=int)



app = tornado.web.Application([
    (r'/upload', coreService.Upload)
])

if __name__ == "__main__":
    tornado.options.parse_command_line()
    server = tornado.httpserver.HTTPServer(app, max_buffer_size=1024*1024*201)
    server.bind(options.port)
    # autodetect cpu cores and fork one process per core
    server.start(0)
    try:        
        print 'running on port %s' % options.port
        tornado.ioloop.IOLoop.instance().start()

    except KeyboardInterrupt:
        tornado.ioloop.IOLoop.instance().stop()

此代码抛出此错误:

File "/opt/tornado/tornado/process.py", line 112, in fork_processes
    raise RuntimeError("Cannot run in multiple processes: IOLoop instance "
RuntimeError: Cannot run in multiple processes: IOLoop instance has already been initialized. You cannot call IOLoop.instance() before calling start_processes()

我只是没有看到它.谢谢

:编辑:

正如Ben所说,我的一种方法给了我麻烦.这是该方法的代码,有人可能从中受益:

from tornado import gen
import motor

db = motor.MotorClient().open_sync().proba
class Upload(BaseHandler):
    @gen.engine
    def post(self):
        fs = yield motor.Op(motor.MotorGridFS(db).open)

        gridin = yield motor.Op(fs.new_file)
        yield motor.Op(gridin.write, 'First part\n')
        yield motor.Op(gridin.write, 'Second part')
        yield motor.Op(gridin.close)

        print gridin._id
        self.write(str(gridin._id))
        self.finish()

编辑2

我找到了问题的最终解决方案.正如本指出的那样,上面的方法给了我麻烦.在Motor文档中记录了包含Motor with Tornado应用程序的正确方法.这是一个为我工作的除外:

if __name__ == "__main__":
    tornado.options.parse_command_line()        
    try:
        server = tornado.httpserver.HTTPServer(app, max_buffer_size=1024*1024*201)
        server.bind(8888)
        server.start(0) # autodetect cpu cores and fork one process per core
        db = motor.MotorClient().open_sync().proba
        print 'running on port %s' % options.port
        # Delayed initialization of settings
        app.settings['db'] = db # from this point on db is available as self.settings['db']
        tornado.ioloop.IOLoop.instance().start()

    except KeyboardInterrupt:
        tornado.ioloop.IOLoop.instance().stop()

Answer 1

使用tornado.web.Application的调试模式会引发此异常.

application = tornado.web.Application([
    (r"/", hello),
],
debug=False)

将debug设置为False以解决此问题.

你可以开始几个过程来监听每个端口:

server = tornado.httpserver.HTTPServer(application)
server.bind(1234)  # port
server.start(4) 
tornado.ioloop.IOLoop.instance().start()

Answer 2

如果我注释掉"核心"和"服务"导入,它会按预期工作.其中一个模块中的某些东西必须初始化单例事件循环(可能是间接的,例如通过创建一个全局的AsyncHTTPClient实例).此警告可以保护您免受父进程中创建的这些对象在子进程中无效的影响.你必须找到创建这些对象的地方(遗憾的是没有好的工具)并在fork之后移动它们,所以它们是在子进程中创建的.