如何使用AWS Elastic Beanstalk运行工作人员？

Question

我正在aws弹性beanstalk上启动一个django应用程序.我想运行后台任务或工作人员以便运行芹菜.

我找不到是否可能.如果是的话怎么可能实现？

这就是我现在正在做的事情,但这每次都会产生一个事件类型错误.

container_commands:
  01_syncdb:
    command: "django-admin.py syncdb --noinput"
    leader_only: true
  50_sqs_email:
    command: "./manage.py celery worker --loglevel=info"
    leader_only: true

Answer 1

正如@ chris-wheadon在评论中建议的那样,你应该尝试在后台运行芹菜作为一个守护神.AWS Elastic Beanstalk 已经使用supervisord来运行一些deamon进程.因此,您可以利用它来运行celeryd并避免为此创建自定义AMI.它对我很好.

我所做的是在EB将应用程序部署到实例后以编程方式将celeryd配置文件添加到实例中.棘手的部分是文件需要为守护程序设置所需的环境变量(例如,如果您在应用程序中使用S3或其他服务,则为AWS访问密钥).

下面是我使用的脚本的副本,将此脚本添加到.ebextensions配置EB环境的文件夹中.

安装脚本在所有EB实例上的/opt/elasticbeanstalk/hooks/appdeploy/post/文件夹(文档)中创建一个文件.其中的任何shell脚本都将在部署后执行.放在那里的shell脚本的工作原理如下:

1. 在celeryenv变量中,virutalenv环境以遵循supervisord符号的格式存储.这是一个以逗号分隔的env变量列表.
2. 然后脚本创建一个变量celeryconf,其中包含配置文件作为字符串,其中包含先前解析的env变量.
3. 然后将此变量传送到一个名为celeryd.confcelery守护程序的supervisord配置文件的文件中.
4. 最后,新创建的配置文件的路径将添加到主supervisord.conf文件中(如果尚未存在).

这是脚本的副本:

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/post/run_supervised_celeryd.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash

      # Get django environment variables
      celeryenv=`cat /opt/python/current/env | tr '\n' ',' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g'`
      celeryenv=${celeryenv%?}

      # Create celery configuraiton script
      celeryconf="[program:celeryd]
      ; Set full path to celery program if using virtualenv
      command=/opt/python/run/venv/bin/celery worker -A myappname --loglevel=INFO

      directory=/opt/python/current/app
      user=nobody
      numprocs=1
      stdout_logfile=/var/log/celery-worker.log
      stderr_logfile=/var/log/celery-worker.log
      autostart=true
      autorestart=true
      startsecs=10

      ; Need to wait for currently executing tasks to finish at shutdown.
      ; Increase this if you have very long running tasks.
      stopwaitsecs = 600

      ; When resorting to send SIGKILL to the program to terminate it
      ; send SIGKILL to its whole process group instead,
      ; taking care of its children as well.
      killasgroup=true

      ; if rabbitmq is supervised, set its priority higher
      ; so it starts first
      priority=998

      environment=$celeryenv"

      # Create the celery supervisord conf script
      echo "$celeryconf" | tee /opt/python/etc/celery.conf

      # Add configuration script to supervisord conf (if not there already)
      if ! grep -Fxq "[include]" /opt/python/etc/supervisord.conf
          then
          echo "[include]" | tee -a /opt/python/etc/supervisord.conf
          echo "files: celery.conf" | tee -a /opt/python/etc/supervisord.conf
      fi

      # Reread the supervisord config
      supervisorctl -c /opt/python/etc/supervisord.conf reread

      # Update supervisord in cache without restarting all services
      supervisorctl -c /opt/python/etc/supervisord.conf update

      # Start/Restart celeryd through supervisord
      supervisorctl -c /opt/python/etc/supervisord.conf restart celeryd

Answer 2

我试图在 PHP 中做类似的事情，但出于某种原因我无法让工作人员运行。我切换到 EC2 服务器上的 AMI，并从那时起就取得了成功。