如何覆盖logstash中来自json的时间戳字段

Question

我有以下类型的json与我一起使用filebeat在弹性搜索中转储 {"@timestamp":"2017-02-10T06:30:51.424Z","beat":{"hostname":"myhostname","name":"mydevice-logger","version":"5.2.0"},"fields":{"device_type":"mydevice","env":"staging"},"metricset":{"module":"system","name":"cpu","rtt":211},"system":{"cpu":{"cores":4,"idle":{"pct":0.000000},"iowait":{"pct":0.000000},"irq":{"pct":0.000000},"nice":{"pct":0.000000},"softirq":{"pct":0.000000},"steal":{"pct":0.000000},"system":{"pct":0.000000},"user":{"pct":0.000000}}},"tags":["automata","box"],"type":"metricbeat-test-log"}

我的logstash(版本5.1.1)配置包含,输入,过滤器和输出如下 -

input { 
  beats {
        port => 5046
        codec => json
  }
}

filter {
    if ...{}
    else if [type] == "metricbeat-test-log" {

      date {
        match => ["@timestamp", "ISO8601"]
      }

      }
    }

}


output {
    if ...{}
    else if [type] == "metricbeat-test-log" {
        stdout { codec => rubydebug   }
    }
} 

类型是正确的但是日期过滤器不起作用.在@timestamp最后还以当前的时间戳始终.我想用@timestampjson中的原始礼物替换它.

Answer 1

我在这个旧线程https://discuss.elastic.co/t/replace-timestamp-with-actual-timestamp-from-log-file/72017之后得到了答案，它解释了"@timestamp":"2017-02-10T06:30:51.424Z"上面的日志行是 JSON 表示@timestamp 字段及其值。按照建议，我在 filebeat 中添加了 json 配置，它对我有用。

- input_type: log
  paths:
    - /var/logs/mylogs/*.log
  fields:
    environment: testing
  document_type: test-metric-document

  json.keys_under_root: true   # added this line 
  json.overwrite_keys: true    # added this line 

虽然我对这个解决方案并不满意，因为我真正需要的是获取时间戳、logstash 事件一个（在其他一些变量中）和来自 @timestamp 的日志的时间戳。