我使用 Google Composer 一段时间了 ( composer-0.5.2-airflow-1.9.0),并且在使用 Airflow 调度程序时遇到了一些问题。调度程序容器有时会崩溃,并且可能会陷入锁定状态,无法启动任何新任务(数据库连接错误),因此我必须重新创建整个 Composer 环境。这次,CrashLoopBackOff调度程序 Pod 无法再重新启动。该错误与我之前遇到的非常相似。这是 Stackdriver 的回溯:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 27, in <module>
args.func(args)
File "/usr/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 826, in scheduler
job.run()
File "/usr/local/lib/python2.7/site-packages/airflow/jobs.py", line 198, in run
self._execute()
File "/usr/local/lib/python2.7/site-packages/airflow/jobs.py", line 1549, in _execute
self._execute_helper(processor_manager)
File "/usr/local/lib/python2.7/site-packages/airflow/jobs.py", line 1594, in _execute_helper
self.reset_state_for_orphaned_tasks(session=session)
File "/usr/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper
result = func(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/airflow/jobs.py", line 266, in reset_state_for_orphaned_tasks
.filter(or_(*filter_for_tis), TI.state.in_(resettable_states))
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2783, in …Run Code Online (Sandbox Code Playgroud)