Celery和Flask在同一个docker-compose中

Question

我正在尝试使用docker-compose我的Flask/Celery/Redis服务.

这是我的docker-compose.yml:

flask:
    build: .
    command: "python3 app.py"
    ports:
        - '5000:5000'
    links:
        - redis
    volumes:
        - .:/usr/src/app:ro

celery:
    build: .
    command: "celery -A app.celery worker --loglevel=info"
    volumes:
        - .:/usr/src/app:ro

redis:
    image: redis
    ports:
        - '6379:6379'

当我运行它时docker-compose,Flask和Redis都可以正常启动并按预期运行.关于Celery,Docker报道:flaskcelery_celery_1 exited with code 1没有其他信息.

如果我在没有Docker的情况下运行我的三个服务,并启动Celery celery -A app.celery worker --loglevel=info,我的应用程序功能就好了.

如有必要,可提供更多信息:

Dockerfile :(此图片requirements.txt也在构建时安装)

FROM python:3.5-onbuild
EXPOSE 5000

requirements.txt:

flask==0.11.1
celery==3.1.23

docker-compose up 输出:

Starting flaskcelery_celery_1
Starting flaskcelery_redis_1
Starting flaskcelery_flask_1
Attaching to flaskcelery_celery_1, flaskcelery_redis_1, flaskcelery_flask_1
redis_1  |                 _._                                                  
redis_1  |            _.-``__ ''-._                                             
redis_1  |       _.-``    `.  `_.  ''-._           Redis 3.2.3 (00000000/0) 64 bit
redis_1  |   .-`` .-```.  ```\/    _.,_ ''-._                                   
redis_1  |  (    '      ,       .-`  | `,    )     Running in standalone mode
redis_1  |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
redis_1  |  |    `-._   `._    /     _.-'    |     PID: 1
redis_1  |   `-._    `-._  `-./  _.-'    _.-'                                   
redis_1  |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
redis_1  |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
redis_1  |   `-._    `-._`-.__.-'_.-'    _.-'                                   
redis_1  |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
redis_1  |  |    `-._`-._        _.-'_.-'    |                                  
redis_1  |   `-._    `-._`-.__.-'_.-'    _.-'                                   
redis_1  |       `-._    `-.__.-'    _.-'                                       
redis_1  |           `-._        _.-'                                           
redis_1  |               `-.__.-'                                               
redis_1  | 
redis_1  | 1:M 23 Aug 10:23:08.409 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1  | 1:M 23 Aug 10:23:08.409 # Server started, Redis version 3.2.3
redis_1  | 1:M 23 Aug 10:23:08.409 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1  | 1:M 23 Aug 10:23:08.409 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1  | 1:M 23 Aug 10:23:08.409 * The server is now ready to accept connections on port 6379
flaskcelery_celery_1 exited with code 1
flask_1  |  * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
flask_1  |  * Restarting with stat
flask_1  |  * Debugger is active!
flask_1  |  * Debugger pin code: 196-119-737

Answer 1

解决了我的问题.我最终发现我可以在Docker镜像上获得命令提示符:

docker build -t <image name> .
docker run -it <image name> /bin/bash

然后尝试celery在容器内运行显示问题:

root@4a6edc5d7372:/usr/src/app# celery -A app.celery worker --loglevel=info
Running a worker with superuser privileges when the
worker accepts messages serialized with pickle is a very bad idea!

If you really want to continue then you have to set the C_FORCE_ROOT
environment variable (but please think about this before you do).

User information: uid=0 euid=0 gid=0 egid=0

Docker通常以root身份运行,出于安​​全原因,Celery不喜欢以root身份运行(我相信你可以通过pickle反序列化来获得代码执行).更安全的解决方案是将celery容器设置为运行nobody.工作docker-compose.yml:

flask:
    build: .
    command: "python3 app.py"
    ports:
        - '5000:5000'
    links:
        - redis
        - celery
    volumes:
        - .:/usr/src/app:ro

celery:
    build: .
    command: "celery -A app.celery worker --loglevel=info"
    user: nobody
    links:
        - redis
    volumes:
        - .:/usr/src/app:ro

redis:
    image: redis
    ports:
        - '6379:6379'

Answer 2

如果您不想处理 docker-compose 而只想使用 docker，那么这可能对您有用。首先，您需要创建一个“supervisord”文件，如下所示。

[supervisord]
nodaemon=true

[program:python]
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
command=python flask_api.py

[program:celeryworker]
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
command=celery -A flask_api worker -c 2

那么你的 docker 文件应该如下所示。显然这只是一个模板，您可以根据需要更改它。

FROM ubuntu:16.04
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential libpq-dev git postgresql-9.5 postgresql-contrib
RUN apt-get install -y supervisor
RUN pip install --upgrade pip
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
WORKDIR /app
CMD ["supervisord"]

这样，您就可以在同一个 docker 容器中运行 Flask 应用程序和 celery Worker。