OSError: [Errno 13] 在 Docker 中初始化 Celery 时权限被拒绝

Question

运行 docker compose 时出现以下错误。该问题在我的 Mac OS 开发环境中根本不会发生（尝试在 Ubuntu 和 debian 中部署时发生此错误），但该错误似乎表明 Celery 无权写入 celerybeat 文件。我已经尝试了几天试图让它工作（试图给芹菜必要的权限），但没有运气。

错误

celery_1    | [2017-06-17 13:08:26,509: INFO/Beat] beat: Starting...
celery_1    | [2017-06-17 13:08:26,556: ERROR/Beat] Removing corrupted schedule file 'celerybeat-schedule': DBAccessError(13, 'Permission denied')
celery_1    | Traceback (most recent call last):
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 376, in setup_schedule
celery_1    |     self._store = self._open_schedule()
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 366, in _open_schedule
celery_1    |     return self.persistence.open(self.schedule_filename, writeback=True)
celery_1    |   File "/usr/local/lib/python2.7/shelve.py", line 243, in open
celery_1    |     return DbfilenameShelf(filename, flag, protocol, writeback)
celery_1    |   File "/usr/local/lib/python2.7/shelve.py", line 227, in __init__
celery_1    |     Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback)
celery_1    |   File "/usr/local/lib/python2.7/anydbm.py", line 85, in open
celery_1    |     return mod.open(file, flag, mode)
celery_1    |   File "/usr/local/lib/python2.7/dbhash.py", line 18, in open
celery_1    |     return bsddb.hashopen(file, flag, mode)
celery_1    |   File "/usr/local/lib/python2.7/bsddb/__init__.py", line 364, in hashopen
celery_1    |     d.open(file, db.DB_HASH, flags, mode)
celery_1    | DBAccessError: (13, 'Permission denied')
celery_1    | [2017-06-17 13:08:26,558: ERROR/Beat] Process Beat
celery_1    | Traceback (most recent call last):
celery_1    |   File "/usr/local/lib/python2.7/site-packages/billiard/process.py", line 292, in _bootstrap
celery_1    |     self.run()
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 553, in run
celery_1    |     self.service.start(embedded_process=True)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 470, in start
celery_1    |     humanize_seconds(self.scheduler.max_interval))
celery_1    |   File "/usr/local/lib/python2.7/site-packages/kombu/utils/__init__.py", line 325, in __get__
celery_1    |     value = obj.__dict__[self.__name__] = self.__get(obj)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 512, in scheduler
celery_1    |     return self.get_scheduler()
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 507, in get_scheduler
celery_1    |     lazy=lazy)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/utils/imports.py", line 53, in instantiate
celery_1    |     return symbol_by_name(name)(*args, **kwargs)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 358, in __init__
celery_1    |     Scheduler.__init__(self, *args, **kwargs)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 185, in __init__
celery_1    |     self.setup_schedule()
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 384, in setup_schedule
celery_1    |     self._store = self._destroy_open_corrupted_schedule(exc)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 371, in _destroy_open_corrupted_schedule
celery_1    |     self._remove_db()
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 363, in _remove_db
celery_1    |     os.remove(self.schedule_filename + suffix)
celery_1    | OSError: [Errno 13] Permission denied: 'celerybeat-schedule'

摘自 Dockerfile

FROM python:2.7-slim
MAINTAINER Maintainer <maintainer@gmail.com>    

RUN apt-get -y install sudo

ENV INSTALL_PATH /minebase
RUN mkdir -p $INSTALL_PATH

WORKDIR $INSTALL_PATH

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .
RUN pip install --editable .

CMD gunicorn -c "python:config.gunicorn" "minebase.app:create_app()"

RUN groupadd -r celery && useradd -r -g celery celery | chpasswd && adduser celery sudo

RUN sudo chown -R celery ./
USER celery

CMD ["celery", "-A", "worker:app", "worker"]

Docker-compose.yml

version: '2'

services:

  postgres:
    image: 'postgres:9.5'
    env_file:
      - '.env'
    volumes:
      - 'postgres:/var/lib/postgresql/data'
    ports:
      - '5432:5432'

  redis:
    image: 'redis:3.0-alpine'
    command: redis-server --requirepass password
    volumes:
      - 'redis:/var/lib/redis/data'
    ports:
      - '6379:6379'

  minebase:
    build: .
    command: >
      gunicorn -c "python:config.gunicorn" --reload "minebase.app:create_app()" --timeout 7200 --workers=5
    env_file:
      - '.env'
    volumes:
      - '.:/minebase'
    ports:
      - '8000:8000'

  nginx:
    restart: always
    build: ./nginx/
    ports:
      - "80:80"
    volumes:
      - '.:/minebase'
    volumes_from:
      - minebase
    links:
      - minebase:minebase
    expose:
      - 80

  celery:
    build: .
    command: celery worker -B -l info -A minebase.blueprints.contact.tasks 
    env_file:
      - '.env'
    volumes:
      - '.:/minebase'

volumes:
  postgres:
  redis:

版本

芹菜版：v3.1.23

Docker 版本：17.05.0-ce

Docker 组合版本：1.13.0

Answer 1

你不需要在 Docker 容器中使用sudo和chown 的所有魔法。

使用现有的非特权用户nobody：

FROM some-base-image
WORKDIR /app
RUN chown nobody:nogroup "celerybeat-schedule"
USER nobody
CMD ["celery", "-A", "your_project.celery_app", "-E", "-B"]

另一种方法是为“celerybeat-schedule”文件指定一个自定义位置。例如，在/tmp目录中：

FROM some-base-image
WORKDIR /app
USER nobody
CMD ["celery", "-A", "your_project.celery_app",, "-E", "-B", \
     "-s", "/tmp/celerybeat-schedule"]

Answer 2

对于那些有同样问题的人来说，可以通过授予 celery 用户对 celerybeat 文件的写访问权限来修复它。我首先通过在 Filezilla 中手动设置文件权限来修复它。如果您想在 dockerfile 中自动执行此操作，可以将此代码附加到文件底部：

USER root

RUN sudo chown -R celery:celery celerybeat-schedule

并确保首先使用以下代码片段安装 sudo：

RUN apt-get -y install sudo