小编Mat*_*ett的帖子

以 JSON 格式格式化气流日志

我需要记录Apache AirflowJSON 格式日志记录到标准输出。Airflow 似乎没有开箱即用地投射此功能。我找到了几个能够完成此任务的 python 模块,但我无法使实现工作。

目前,我正在应用一个类airflow/utils/logging.py来修改记录器,如下所示:

from pythonjsonlogger import jsonlogger

class StackdriverJsonFormatter(jsonlogger.JsonFormatter, object):
def __init__(self, fmt="%(levelname) %(asctime) %(nanotime) %(severity) %(message)", style='%', *args, **kwargs):
    jsonlogger.JsonFormatter.__init__(self, fmt=fmt, *args, **kwargs)

def process_log_record(self, log_record):
    if log_record.get('level'):
        log_record['severity'] = log_record['level']
        del log_record['level']
    else: 
        log_record['severity'] = log_record['levelname']
        del log_record['levelname']
    if log_record.get('asctime'):
        log_record['timestamp'] = log_record['asctime']
        del log_record['asctime']
    now = datetime.datetime.now().strftime('%Y-%m-%dT%H:%M:%S.%fZ')
    log_record['nanotime'] = now
    return super(StackdriverJsonFormatter, self).process_log_record(log_record)
Run Code Online (Sandbox Code Playgroud)

我正在执行此代码,/airflow/settings.py如下所示:

from airflow.utils import logging as logconf

def configure_logging(log_format=LOG_FORMAT):
     handler = logconf.logging.StreamHandler(sys.stdout)
     formatter = …
Run Code Online (Sandbox Code Playgroud)

python logging airflow python-jsonschema

6
推荐指数
1
解决办法
2841
查看次数

标签 统计

airflow ×1

logging ×1

python ×1

python-jsonschema ×1