Logstash用syslog日期替换@timestamp

Question

我有点困惑.我正试图拔出syslog date (backfilling the logstash)并替换@timestamp它.我几乎尝试了一切.

这是我的过滤器

filter {
   if [type] == "syslog" {
   grok {
     match => {
"message" => ["%{SYSLOGTIMESTAMP:DATETIME} %{WORD:SERVER} (?<BINARY>(.*?)(php\-cgi|php))\: %{DATA:PHP_ERROR_TYPE}\:\s\s(?<PHP_ERROR_DESC>(.*?)(e\s\d))""]
  }
}

date {
  match => { "DATETIME" => [ "MMM  d HH:mm:ss", "MMM dd HH:mm:ss", "ISO8601" ] }
  target => "@timestamp"
  add_tag => [ "tmatch" ]
}

if !("_grokparsefailure" in [tags]) {
  mutate {
    replace => [ "@source_host", "%{SERVER}" ]
  }
}
mutate {
  remove_field => [ "SERVER" ]
}
}
}

样本输出:

{
    "message" => "Sep 10 00:00:00 xxxxxxx",
    "@timestamp" => "2013-12-05T13:29:35.169Z",
      "@version" => "1",
          "type" => "xxxx",
          "host" => "127.0.0.1:xxx",
      "DATETIME" => "Sep 10 00:00:00",
        "BINARY" => "xxxx",
"PHP_ERROR_TYPE" => "xxxx",
"PHP_ERROR_DESC" => "xxxxx",
          "tags" => [
    [0] "tmatch"
],
  "@source_host" => "xxx"
}

tmatch在标签中,所以我假设日期过滤器有效,但为什么我仍然有:

@timestamp => "2013-12-05T13:29:35.169Z"

？

谢谢你的帮助(我的logstash是logstash-1.2.2-flatjar.jar)

Answer 1

我们来看看你的日期过滤器:

date {
  match => { "DATETIME" => [ "MMM  d HH:mm:ss", "MMM dd HH:mm:ss", "ISO8601" ] }
  target => "@timestamp"
  add_tag => [ "tmatch" ]
}

特别是匹配参数:

match => { "DATETIME" => [ "MMM  d HH:mm:ss", "MMM dd HH:mm:ss", "ISO8601" ] }

匹配期望一个数组.我不确定你传递的是什么,但它肯定不是一个阵列.我尝试使用它-v,我很惊讶地看到它没有抱怨.

你可能意味着更接近这个:

match => ["DATETIME", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss", "ISO8601"]

注意,数组的第一个元素是目标字段; 其他元素是要匹配的模式.

过去,你真的只需要传递你期望的一种格式,但看起来它包含在你发送的三种格式中.

Answer 2

如果您希望时间戳显示为您的时区格式,而不是UTC时间,您可以这样做

ruby {
    code => "event['@timestamp'] = event['@timestamp'].local('-08:00')"
}

之前:@timestamp => "2013-12-05T13:29:35.169Z"

之后:@timestamp => "2013-12-05T05:29:35.169-08:00"

更新:本地方法无法在1.4.2版中运行.所以,改变另一个API

ruby {
    code => "event['@timestamp'] = event['@timestamp'].getlocal"
}