相关疑难解决方法(0)

AirflowException:Celery 命令失败 - 记录的主机名与此实例的主机名不匹配

我在两个 AWS EC2 实例上运行的集群环境中运行 Airflow。一份给主人,一份给工人。不过,工作节点在运行“$airflowworker”时会定期抛出此错误:

[2018-08-09 16:15:43,553] {jobs.py:2574} WARNING - The recorded hostname ip-1.2.3.4 does not match this instance's hostname ip-1.2.3.4.eco.tanonprod.comanyname.io
Traceback (most recent call last):
  File "/usr/bin/airflow", line 27, in <module>
    args.func(args)
  File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 387, in run
    run_job.run()
  File "/usr/local/lib/python3.6/site-packages/airflow/jobs.py", line 198, in run
    self._execute()
  File "/usr/local/lib/python3.6/site-packages/airflow/jobs.py", line 2527, in _execute
    self.heartbeat()
  File "/usr/local/lib/python3.6/site-packages/airflow/jobs.py", line 182, in heartbeat
    self.heartbeat_callback(session=session)
  File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 50, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/airflow/jobs.py", line 2575, in heartbeat_callback
    raise AirflowException("Hostname of …
Run Code Online (Sandbox Code Playgroud)

airflow airflow-scheduler

8
推荐指数
2
解决办法
1万
查看次数

如何从github安装Apache Airflow

问题:我想apache-airflow在Github上使用最新版本的Apache-Airflow以及所有依赖项进行安装吗?

我该如何使用pip

在生产环境中使用它也安全吗?

airflow apache-airflow

4
推荐指数
1
解决办法
2433
查看次数

气流1.10安装失败

我有一个使用Airflow版本1.9的Airflow环境,该环境在Amazon EC2实例上运行.我需要升级到Airflow的最新版本1.10.我可以选择从1.9版升级或在新服务器上新安装1.10.气流版本1.10未在Pip上列出,所以我通过此命令从Git安装它,

pip-3.6 install git+git://github.com/apache/incubator-airflow.git@v1-10-stable
Run Code Online (Sandbox Code Playgroud)

此命令成功安装Airflow版本1.10.您可以通过运行命令airflow version并查看输出来查看

  ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
   v1.10.0
Run Code Online (Sandbox Code Playgroud)

当我尝试启动Airflow调度程序时,airflow scheduler我得到以下异常,

ModuleNotFoundError: No module named 'MySQLdb'
[2018-08-14 14:03:16,195] {celery_executor.py:112} ERROR - Error syncing the celery executor, ignoring it:
[2018-08-14 14:03:16,195] {celery_executor.py:113} ERROR - …
Run Code Online (Sandbox Code Playgroud)

pip airflow airflow-scheduler

1
推荐指数
1
解决办法
4365
查看次数

标签 统计

airflow ×3

airflow-scheduler ×2

apache-airflow ×1

pip ×1