celery worker 仅在未分离时导入任务

Question

我正在尝试让我的 django 应用程序将任务提交给 celery 工作人员，并且当工作人员附加运行时它成功了。一旦我添加 --detach 任务就无法注册。

[2020-10-26 04:09:33,159: ERROR/MainProcess] Received unregistered task of type 'devapp.tasks.create_random_files'.
The message has been ignored and discarded.

Did you remember to import the module containing this task?
Or maybe you're using relative imports?

Please see
http://docs.celeryq.org/en/latest/internals/protocol.html
for more information.

The full contents of the message body was:
'[[20, "blah", 5120], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' (93b)
Traceback (most recent call last):
  File "/pysite/project/venv/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 562, in on_task_received
    strategy = strategies[type_]
KeyError: 'devapp.tasks.create_random_files'

在我的 tasks.py 我有

import os
import string
import subprocess
from celery import shared_task, task

@shared_task(name='devapp.tasks.create_random_files')
def create_random_files(total,filename,size):
    for i in range(int(total)):
        filenum = str(i).zfill(5)
        rnd_filename = '/brdis/{}-{}'.format(filenum,filename)
        with open(rnd_filename, 'wb') as f:
            f.write(os.urandom(size))
            f.close
    return '{} random files created'.format(total)

在我的 views.py 中我有

from django.shortcuts import render
from django.http import HttpResponse

from django.contrib import messages
from django.views.generic import TemplateView
from django.views.generic.list import ListView
from django.views.generic.edit import FormView
from django.shortcuts import redirect

from .forms import CreateRandomFilesForm, GetFileListForm, ClamScanFileForm
from .tasks import create_random_files, clamscanfile

class ClamScanFileView(FormView):
    template_name = 'devapp/clamscan_file.html'
    form_class = ClamScanFileForm

    def form_valid(self, form):
        filename = form.cleaned_data.get('filename')
        clamscanfile.delay(filename)
        messages.success(self.request, 'We are scanning your file!s Wait a moment and refresh this page.')
        return redirect('files_list')

我知道它可能是相对导入（我需要了解的其他东西）但我不明白为什么将 --detach 添加到 celery worker 命令会产生此错误：

celery -A project worker -E --loglevel=INFO --logfile=/var/log/celery/celeryd.log --pidfile=/var/run/celery/celeryd.pid

工作正常，但...

celery -A project worker -E --loglevel=INFO --logfile=/var/log/celery/celeryd.log --pidfile=/var/run/celery/celeryd.pid --detach

启动工作人员但不注册任务？

任何帮助表示赞赏。

FWIW

celery report

software -> celery:4.4.7 (cliffs) kombu:4.6.11 py:3.6.8
            billiard:3.6.3.0 py-amqp:2.6.1
platform -> system:Linux arch:64bit, ELF
            kernel version:3.10.0-1127.19.1.el7.x86_64 imp:CPython
loader   -> celery.loaders.default.Loader
settings -> transport:amqp results:disabled

Answer 1

这是 4.4.7 的一个错误。降级到 4.4.6 或升级到 5 应该可以解决这个问题。

https://github.com/celery/celery/issues/6370