我试图解析tomcat日志并将输出传递给弹性搜索.或多或少它运作良好.当我看到弹性搜索索引数据时,它包含大量具有标签字段的匹配数据_grokparsefailure.这导致了大量重复的匹配数据.为了避免这种情况,我试图在标签包含时删除事件_grokparsefailure.此配置写在grok filter下面的logstash.conf文件中.仍然输出到弹性搜索包含索引的doc包含标签_grokparsefailure.如果grok失败,我不希望该匹配转到弹性搜索,因为它在弹性搜索中导致重复数据.
logstash.conf 文件是:
input {
file {
path => "/opt/elasticSearch/logstash-1.4.2/input.log"
codec => multiline {
pattern => "^\["
negate => true
what => previous
}
start_position => "end"
}
}
filter {
grok {
match => [
"message", "^\[%{GREEDYDATA}\] %{GREEDYDATA} Searching hotels for country %{GREEDYDATA:country}, city %{GREEDYDATA:city}, checkin %{GREEDYDATA:checkin}, checkout %{GREEDYDATA:checkout}, roomstay %{GREEDYDATA:roomstay}, No. of hotels returned is %{NUMBER:hotelcount} ."
]
}
if "_grokparsefailure" in [tags]{
drop { }
}
}
output {
file …