小编Dal*_*lar的帖子

Airflow Scheduler 不断崩溃,数据库连接错误(Google Composer)

我使用 Google Composer 一段时间了 ( composer-0.5.2-airflow-1.9.0),并且在使用 Airflow 调度程序时遇到了一些问题。调度程序容器有时会崩溃,并且可能会陷入锁定状态,无法启动任何新任务(数据库连接错误),因此我必须重新创建整个 Composer 环境。这次,CrashLoopBackOff调度程序 Pod 无法再重新启动。该错误与我之前遇到的非常相似。这是 Stackdriver 的回溯:

Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 27, in <module>
    args.func(args)
  File "/usr/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 826, in scheduler
    job.run()
  File "/usr/local/lib/python2.7/site-packages/airflow/jobs.py", line 198, in run
    self._execute()
  File "/usr/local/lib/python2.7/site-packages/airflow/jobs.py", line 1549, in _execute
    self._execute_helper(processor_manager)
  File "/usr/local/lib/python2.7/site-packages/airflow/jobs.py", line 1594, in _execute_helper
    self.reset_state_for_orphaned_tasks(session=session)
  File "/usr/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/airflow/jobs.py", line 266, in reset_state_for_orphaned_tasks
    .filter(or_(*filter_for_tis), TI.state.in_(resettable_states))
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2783, in …
Run Code Online (Sandbox Code Playgroud)

airflow airflow-scheduler google-cloud-composer

5
推荐指数
1
解决办法
4024
查看次数