Logstash解析csv日期

Question

我有一个问题从csv解析日期,我找不到问题(人们会认为)简单日期 - 日/月/日.这是我的csv文件的结构:

Date,Key-values,Line Item,Creative,Ad unit,Creative size,Ad server impressions,Ad server clicks,Ad server CTR
04/04/16,prid=DUBAP,Hilton_PostAuth 1,Stop Clicking Around - 300x250,383UKHilton_300x250,300 x 250,31,0,0.00%
04/04/16,prid=DUBAP,Hilton_PostAuth 2,16-0006_Auction_Banners_300x250_cat4,383UKHilton_300x250,300 x 250,59,0,0.00%

和我的logstash.config文件:

input {
  file {
    path => "/Users/User/*.csv"
    type => "core2"
    start_position => "beginning"    
  }
}
filter {
  csv {
      columns => ["Date","Key-values","Line Item","Creative","Ad unit","Creative size","Ad server impressions","Ad server clicks","Ad server CTR"]
      separator => ","
  }
  date {
    match => ["Date", "dd/MM/YY"]
  }
  mutate {convert => ["Ad server impressions", "float"]}
  mutate {convert => ["Ad server clicks", "float"]}
  mutate {convert => ["Ad server CTR", "float"]}
}
output {  
    elasticsearch {
        action => "index"
        hosts => "localhost"
        index => "test1"
        workers => 1
    }
    stdout {}
}

我也尝试过将日期设为"dd/MM/yy"而没有运气的组合,Date没有被编入日期索引,我只能在Kibana中选择@timestamp ..

我认为必须有一个简单的事情,我只是错过了,但至于这一刻我找不到它..

干杯!

编辑1:

当logstash启动时以及如何处理数据时,请查找我的控制台输出:

Settings: Default pipeline workers: 4
Pipeline main started
Failed parsing date from field {:field=>"Date", :value=>"Date", :exception=>"Invalid format: \"Date\"", :config_parsers=>"dd/MM/YY", :config_locale=>"default=en_US", :level=>:warn}
2016-05-06T20:32:48.034Z Pawels-MacBook-Air.local Date,Key-values,Line Item,Creative,Ad unit,Creative size,Ad server impressions,Ad server clicks,Ad server CTR
2016-04-03T23:00:00.000Z Pawels-MacBook-Air.local 04/04/16,prid=DUBAP,Hilton_PostAuth 1,Stop Clicking Around - 300x250,383UKHilton_300x250,300 x 250,31,0,0.00%

它仍然将它加载到Elasticsearch中,但在Kibana中没有'Date'字段 - 我只能使用@timestamp

干杯

Answer 1

实际上date过滤器的作用是:

日期过滤器用于解析字段中的日期,然后使用该日期或时间戳作为事件的logstash时间戳.

因此,使用该配置,它会读取您的日期并将其用作时间戳字段.如果要将其用作单独的字段,请配置为:

date {
    match => ["Date", "dd/MM/yy"]
    target => "Date"
}