如何减少 Logstash 的 RAM 使用量？

Question

我正在寻找 Logstash RAM 问题的答案，因为它几乎 100%。我进行了很多搜索，但它们并不适合我。下面的代码是我的logstash.conf 文件。我认为这需要一些小改动。

Logstash.conf：

input {
  file {
        path => ["c:/mylogs/*.txt"]
        start_position => "beginning" 
        discover_interval => 10
        stat_interval => 10
        sincedb_write_interval => 10
        close_older => 10
        codec => "json"
  }
}

filter {
  date {
        match => ["mydate","yyyy-MM-dd HH:mm:ss.SSSS" ]
        timezone => "UTC"  
  }
  date {
        match => ["TimeStamp", "ISO8601"]
  }

  json {
        source => "request"
        target => "parsedJson"
  }
}

output {
  stdout {
        codec => rubydebug
  }
  elasticsearch {
        hosts => [ "http://localhost:9200" ]
        index => "log-%{+YYYY.MM}"
  }
}

Answer 1

设置 Logstash JVM 选项

imply setting LS_JAVA_OPTS will not work because that only appends to the default Logstash JVM Options and does not replace them. You can set JAVA_OPTS but that will not work unless you also est HEAP_DUMP_PATH. Below is the final configuration we use.

# replace logstash Java options since we only have 4 cores in production
# you will get a waring in the logs
# LS_JAVA_OPTS only appends
export JAVA_OPTS="-XX:+UseSerialGC -Djava.awt.headless=true -XX:+HeapDumpOnOutOfMemoryError"
# logstash will append Xmx
export LS_HEAP_SIZE="1g"
# this needs to be set because logstash will always append it
# if it's missing you will get an empty VM argument and it won't start
export HEAP_DUMP_PATH="-XX:HeapDumpPath=$LOGSTASH/heapdump.hprof"