Gunicorn 工人以信号 9 终止

Question

我正在运行一个 Flask 应用程序，并将它从 Docker 容器托管在 Kubernetes 上。Gunicorn 正在管理回复 API 请求的工作人员。

以下警告消息是经常发生的，似乎由于某种原因请求被取消。在 Kubernetes 上，pod 没有表现出任何奇怪的行为或重新启动并保持在其内存和 CPU 限制的 80% 以内。

[2021-03-31 16:30:31 +0200] [1] [WARNING] Worker with pid 26 was terminated due to signal 9

我们如何才能找出这些工人被杀害的原因？

Answer 1

在我们的案例中，应用程序需要大约 5-7 分钟才能将 ML 模型和字典加载到内存中。所以添加 600 秒的超时时间就解决了我们的问题。

gunicorn main:app \
   --workers 1 \
   --worker-class uvicorn.workers.UvicornWorker \
   --bind 0.0.0.0:8443 \
   --timeout 600

Answer 2

I encountered the same warning message.

[WARNING] Worker with pid 71 was terminated due to signal 9

I came across this faq, which says that "A common cause of SIGKILL is when OOM killer terminates a process due to low memory condition."

I used dmesg realized that indeed it was killed because it was running out of memory.

Out of memory: Killed process 776660 (gunicorn)