气流网络服务器突然停止启动

Question

我的气流网络服务器突然停止启动。当我尝试启动网络服务器时，它没有提供 UI。

我尝试将 db 重置为airflow resetdb并airflow initdb重新启动所有服务。降级 Gunicorn 并再次升级。然而，重新启动我的 linux 机器，没有任何改变。

网络服务器的日志如下：

[2019-05-17 08:08:00 +0000] [14978] [INFO] Starting gunicorn 19.9.0
[2019-05-17 08:08:00 +0000] [14978] [INFO] Listening at: http://0.0.0.0:8081 (14978)
[2019-05-17 08:08:00 +0000] [14978] [INFO] Using worker: sync
[2019-05-17 08:08:00 +0000] [14983] [INFO] Booting worker with pid: 14983
[2019-05-17 08:08:00 +0000] [14984] [INFO] Booting worker with pid: 14984
[2019-05-17 08:08:00 +0000] [14985] [INFO] Booting worker with pid: 14985
[2019-05-17 08:08:00 +0000] [14986] [INFO] Booting worker with pid: 14986
[2019-05-17 08:08:02,179] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-05-17 08:08:02,279] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-05-17 08:08:02,324] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-05-17 08:08:02,342] {models.py:273} INFO - Filling up the DagBag from /root/airflow/dags
[2019-05-17 08:08:02,376] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-05-17 08:08:02,435] {models.py:273} INFO - Filling up the DagBag from /root/airflow/dags
[2019-05-17 08:08:02,521] {models.py:273} INFO - Filling up the DagBag from /root/airflow/dags
[2019-05-17 08:08:02,524] {models.py:273} INFO - Filling up the DagBag from /root/airflow/dags
[2019-05-17 08:10:00 +0000] [14978] [CRITICAL] WORKER TIMEOUT (pid:14984)
[2019-05-17 08:10:00 +0000] [14978] [CRITICAL] WORKER TIMEOUT (pid:14985)
[2019-05-17 08:10:00 +0000] [14978] [CRITICAL] WORKER TIMEOUT (pid:14986)
[2019-05-17 08:10:00 +0000] [14978] [CRITICAL] WORKER TIMEOUT (pid:14983)
[2019-05-17 08:10:01 +0000] [15161] [INFO] Booting worker with pid: 15161
[2019-05-17 08:10:01 +0000] [15164] [INFO] Booting worker with pid: 15164
[2019-05-17 08:10:01 +0000] [15167] [INFO] Booting worker with pid: 15167
[2019-05-17 08:10:01 +0000] [15168] [INFO] Booting worker with pid: 15168
[2019-05-17 08:10:03,953] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-05-17 08:10:04,007] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-05-17 08:10:04,020] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-05-17 08:10:04,036] {__init__.py:51} INFO - Using executor LocalExecutor

有没有人遇到过同样的问题？或者你有什么建议吗？

Answer 1

这是一个对我有用的可能解决方案。

确保dags_folder不包含任何与您的 dags 定义和配置不相关的文件。

Airflow 网络服务器定期扫描dag_folder，我发现如果该文件夹非常大，扫描会导致服务器停止运行。

希望这对您有帮助:)