小编Vil*_*mar的帖子

Python Redis Queue(rq) - 如何避免为每个作业预加载ML模型?

我想用rq排队我的ml预测.示例代码(pesudo-ish):

predict.py:

import tensorflow as tf

def predict_stuff(foo):
    model = tf.load_model()
    result = model.predict(foo)
    return result
Run Code Online (Sandbox Code Playgroud)

app.py:

from rq import Queue
from redis import Redis
from predict import predict_stuff

q = Queue(connection=Redis())
for foo in baz:
    job = q.enqueue(predict_stuff, foo)
Run Code Online (Sandbox Code Playgroud)

worker.py:

import sys
from rq import Connection, Worker

# Preload libraries
import tensorflow as tf

with Connection():
    qs = sys.argv[1:] or ['default']

    w = Worker(qs)
    w.work()
Run Code Online (Sandbox Code Playgroud)

我已经阅读了rq文档,解释说你可以预加载库以避免每次运行作业时都导入它们(因此在示例代码中我在worker代码中导入tensorflow).但是,我还希望移动模型加载,predict_stuff以避免每次工作人员运行作业时加载模型.我该怎么办呢?

python redis python-rq

12
推荐指数
2
解决办法
463
查看次数

标签 统计

python ×1

python-rq ×1

redis ×1