Logstash conf 错误 - amazon_es

Ant*_*ert 2 amazon-web-services elasticsearch logstash aws-elasticsearch

我第一次尝试配置我的 logstash.conf 文件并输出到 amazon_es。

我的整个logstash.conf文件在这里:

input {
  jdbc { 
    jdbc_connection_string => "jdbc:mysql://localhost:3306/testdb"
    # The user we wish to execute our statement as
    jdbc_user => "root"
    jdbc_password => "root"
    # The path to our downloaded jdbc driver
    jdbc_driver_library => "/mnt/c/Users/xxxxxxxx/mysql-connector-java-5.1.45/mysql-connector-java-5.1.45-bin.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    # our query
    statement => "SELECT * FROM testtable"
    }
  }
output {
  amazon_es {
    hosts => ["search-xxxxx.eu-west-3.es.amazonaws.com"]
    region => "eu-west-3"
    aws_access_key_id => 'xxxxxxxxxxxxxxxxxxxxxx'
    aws_secret_access_key => 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
    index => "test-migrate"
    document_type => "data"
    }
}
Run Code Online (Sandbox Code Playgroud)

我从我的数据库中选择了 3 个元素,但是第一次运行脚本时,只有第一个元素在弹性搜索中被索引。我第二次运行它时,所有 3 个元素都被索引。每次我使用这个 conf 文件运行 logstash 时,我都会收到错误消息。

编辑2:

    [2018-02-08T14:31:18,270][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/modules/fb_apache/configuration"}
    [2018-02-08T14:31:18,279][DEBUG][logstash.plugins.registry] Adding plugin to the registry {:name=>"fb_apache", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x47c515a1 @module_name="fb_apache", @directory="/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/modules/fb_apache/configuration", @kibana_version_parts=["6", "0", "0"]>}
    [2018-02-08T14:31:18,286][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/modules/netflow/configuration"}
    [2018-02-08T14:31:18,287][DEBUG][logstash.plugins.registry] Adding plugin to the registry {:name=>"netflow", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x6f1a5910 @module_name="netflow", @directory="/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/modules/netflow/configuration", @kibana_version_parts=["6", "0", "0"]>}
    [2018-02-08T14:31:18,765][DEBUG][logstash.runner          ] -------- Logstash Settings (* means modified) ---------
    [2018-02-08T14:31:18,765][DEBUG][logstash.runner          ] node.name: "DEVFE-AMT"
    [2018-02-08T14:31:18,766][DEBUG][logstash.runner          ] *path.config: "logstash.conf"
    [2018-02-08T14:31:18,766][DEBUG][logstash.runner          ] path.data: "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/data"
    [2018-02-08T14:31:18,767][DEBUG][logstash.runner          ] modules.cli: []
    [2018-02-08T14:31:18,768][DEBUG][logstash.runner          ] modules: []
    [2018-02-08T14:31:18,768][DEBUG][logstash.runner          ] modules_setup: false
    [2018-02-08T14:31:18,768][DEBUG][logstash.runner          ] config.test_and_exit: false
    [2018-02-08T14:31:18,769][DEBUG][logstash.runner          ] config.reload.automatic: false
    [2018-02-08T14:31:18,769][DEBUG][logstash.runner          ] config.reload.interval: 3000000000
    [2018-02-08T14:31:18,769][DEBUG][logstash.runner          ] config.support_escapes: false
    [2018-02-08T14:31:18,770][DEBUG][logstash.runner          ] metric.collect: true
    [2018-02-08T14:31:18,770][DEBUG][logstash.runner          ] pipeline.id: "main"
    [2018-02-08T14:31:18,771][DEBUG][logstash.runner          ] pipeline.system: false
    [2018-02-08T14:31:18,771][DEBUG][logstash.runner          ] pipeline.workers: 8
    [2018-02-08T14:31:18,771][DEBUG][logstash.runner          ] pipeline.output.workers: 1
    [2018-02-08T14:31:18,772][DEBUG][logstash.runner          ] pipeline.batch.size: 125
    [2018-02-08T14:31:18,772][DEBUG][logstash.runner          ] pipeline.batch.delay: 50
    [2018-02-08T14:31:18,772][DEBUG][logstash.runner          ] pipeline.unsafe_shutdown: false
    [2018-02-08T14:31:18,772][DEBUG][logstash.runner          ] pipeline.java_execution: false
    [2018-02-08T14:31:18,773][DEBUG][logstash.runner          ] pipeline.reloadable: true
    [2018-02-08T14:31:18,773][DEBUG][logstash.runner          ] path.plugins: []
    [2018-02-08T14:31:18,773][DEBUG][logstash.runner          ] config.debug: false
    [2018-02-08T14:31:18,776][DEBUG][logstash.runner          ] *log.level: "debug" (default: "info")
    [2018-02-08T14:31:18,783][DEBUG][logstash.runner          ] version: false
    [2018-02-08T14:31:18,784][DEBUG][logstash.runner          ] help: false
    [2018-02-08T14:31:18,784][DEBUG][logstash.runner          ] log.format: "plain"
    [2018-02-08T14:31:18,786][DEBUG][logstash.runner          ] http.host: "127.0.0.1"
    [2018-02-08T14:31:18,793][DEBUG][logstash.runner          ] http.port: 9600..9700
    [2018-02-08T14:31:18,793][DEBUG][logstash.runner          ] http.environment: "production"
    [2018-02-08T14:31:18,794][DEBUG][logstash.runner          ] queue.type: "memory"
    [2018-02-08T14:31:18,796][DEBUG][logstash.runner          ] queue.drain: false
    [2018-02-08T14:31:18,804][DEBUG][logstash.runner          ] queue.page_capacity: 67108864
    [2018-02-08T14:31:18,809][DEBUG][logstash.runner          ] queue.max_bytes: 1073741824
    [2018-02-08T14:31:18,822][DEBUG][logstash.runner          ] queue.max_events: 0
    [2018-02-08T14:31:18,823][DEBUG][logstash.runner          ] queue.checkpoint.acks: 1024
    [2018-02-08T14:31:18,836][DEBUG][logstash.runner          ] queue.checkpoint.writes: 1024
    [2018-02-08T14:31:18,837][DEBUG][logstash.runner          ] queue.checkpoint.interval: 1000
    [2018-02-08T14:31:18,846][DEBUG][logstash.runner          ] dead_letter_queue.enable: false
    [2018-02-08T14:31:18,854][DEBUG][logstash.runner          ] dead_letter_queue.max_bytes: 1073741824
    [2018-02-08T14:31:18,859][DEBUG][logstash.runner          ] slowlog.threshold.warn: -1
    [2018-02-08T14:31:18,868][DEBUG][logstash.runner          ] slowlog.threshold.info: -1
    [2018-02-08T14:31:18,873][DEBUG][logstash.runner          ] slowlog.threshold.debug: -1
    [2018-02-08T14:31:18,885][DEBUG][logstash.runner          ] slowlog.threshold.trace: -1
    [2018-02-08T14:31:18,887][DEBUG][logstash.runner          ] keystore.classname: "org.logstash.secret.store.backend.JavaKeyStore"
    [2018-02-08T14:31:18,896][DEBUG][logstash.runner          ] keystore.file: "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/config/logstash.keystore"
    [2018-02-08T14:31:18,896][DEBUG][logstash.runner          ] path.queue: "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/data/queue"
    [2018-02-08T14:31:18,911][DEBUG][logstash.runner          ] path.dead_letter_queue: "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/data/dead_letter_queue"
    [2018-02-08T14:31:18,911][DEBUG][logstash.runner          ] path.settings: "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/config"
    [2018-02-08T14:31:18,926][DEBUG][logstash.runner          ] path.logs: "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/logs"
    [2018-02-08T14:31:18,926][DEBUG][logstash.runner          ] --------------- Logstash Settings -------------------
    [2018-02-08T14:31:18,998][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
    [2018-02-08T14:31:19,067][DEBUG][logstash.agent           ] Setting up metric collection
    [2018-02-08T14:31:19,147][DEBUG][logstash.instrument.periodicpoller.os] Starting {:polling_interval=>5, :polling_timeout=>120}
    [2018-02-08T14:31:19,293][DEBUG][logstash.instrument.periodicpoller.jvm] Starting {:polling_interval=>5, :polling_timeout=>120}
    [2018-02-08T14:31:19,422][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
    [2018-02-08T14:31:19,429][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
    [2018-02-08T14:31:19,453][DEBUG][logstash.instrument.periodicpoller.persistentqueue] Starting {:polling_interval=>5, :polling_timeout=>120}
    [2018-02-08T14:31:19,464][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] Starting {:polling_interval=>5, :polling_timeout=>120}
    [2018-02-08T14:31:19,519][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.2.0"}
    [2018-02-08T14:31:19,537][DEBUG][logstash.agent           ] Starting agent
    [2018-02-08T14:31:19,565][DEBUG][logstash.agent           ] Starting puma
    [2018-02-08T14:31:19,580][DEBUG][logstash.agent           ] Trying to start WebServer {:port=>9600}
    [2018-02-08T14:31:19,654][DEBUG][logstash.config.source.local.configpathloader] Skipping the following files while reading config since they don't match the specified glob pattern {:files=>["/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/CONTRIBUTORS", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/Gemfile", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/Gemfile.lock", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/LICENSE", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/NOTICE.TXT", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/bin", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/config", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/data", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/lib", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/logs", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/logstash-core", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/logstash-core-plugin-api", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/modules", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/tools", "/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/vendor"]}
    [2018-02-08T14:31:19,658][DEBUG][logstash.api.service     ] [api-service] start
    [2018-02-08T14:31:19,662][DEBUG][logstash.config.source.local.configpathloader] Reading config file {:config_file=>"/mnt/c/Users/anthony.maffert/l/logstash-6.2.0/logstash.conf"}
    [2018-02-08T14:31:19,770][DEBUG][logstash.agent           ] Converging pipelines state {:actions_count=>1}
    [2018-02-08T14:31:19,776][DEBUG][logstash.agent           ] Executing action {:action=>LogStash::PipelineAction::Create/pipeline_id:main}
    [2018-02-08T14:31:19,948][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
    [2018-02-08T14:31:21,157][DEBUG][logstash.plugins.registry] On demand adding plugin to the registry {:name=>"jdbc", :type=>"input", :class=>LogStash::Inputs::Jdbc}
    [2018-02-08T14:31:21,557][DEBUG][logstash.plugins.registry] On demand adding plugin to the registry {:name=>"plain", :type=>"codec", :class=>LogStash::Codecs::Plain}
    [2018-02-08T14:31:21,580][DEBUG][logstash.codecs.plain    ] config LogStash::Codecs::Plain/@id = "plain_32fc0754-0187-437b-9d4d-2611eaba9a45"
    [2018-02-08T14:31:21,581][DEBUG][logstash.codecs.plain    ] config LogStash::Codecs::Plain/@enable_metric = true
    [2018-02-08T14:31:21,581][DEBUG][logstash.codecs.plain    ] config LogStash::Codecs::Plain/@charset = "UTF-8"
    [2018-02-08T14:31:21,612][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@jdbc_connection_string = "jdbc:mysql://localhost:3306/testdb"
    [2018-02-08T14:31:21,613][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@jdbc_user = "root"
    [2018-02-08T14:31:21,616][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@jdbc_password = <password>
    [2018-02-08T14:31:21,623][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@jdbc_driver_library = "/mnt/c/Users/anthony.maffert/Desktop/DocumentsUbuntu/mysql-connector-java-5.1.45/mysql-connector-java-5.1.45-bin.jar"
    [2018-02-08T14:31:21,624][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@jdbc_driver_class = "com.mysql.jdbc.Driver"
    [2018-02-08T14:31:21,631][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@statement = "SELECT * FROM testtable"
    [2018-02-08T14:31:21,633][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@id = "ff7529f734e0813846bc8e3b2bcf0794d99ff5cb61b947e0497922b083b3851a"
    [2018-02-08T14:31:21,647][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@enable_metric = true
    [2018-02-08T14:31:21,659][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@codec = <LogStash::Codecs::Plain id=>"plain_32fc0754-0187-437b-9d4d-2611eaba9a45", enable_metric=>true, charset=>"UTF-8">
    [2018-02-08T14:31:21,663][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@add_field = {}
    [2018-02-08T14:31:21,663][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@jdbc_paging_enabled = false
    [2018-02-08T14:31:21,678][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@jdbc_page_size = 100000
    [2018-02-08T14:31:21,679][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@jdbc_validate_connection = false
    [2018-02-08T14:31:21,693][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@jdbc_validation_timeout = 3600
    [2018-02-08T14:31:21,694][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@jdbc_pool_timeout = 5
    [2018-02-08T14:31:21,708][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@sequel_opts = {}
    [2018-02-08T14:31:21,708][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@sql_log_level = "info"
    [2018-02-08T14:31:21,715][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@connection_retry_attempts = 1
    [2018-02-08T14:31:21,716][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@connection_retry_attempts_wait_time = 0.5
    [2018-02-08T14:31:21,721][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@parameters = {}
    [2018-02-08T14:31:21,723][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@last_run_metadata_path = "/home/maffer_a/.logstash_jdbc_last_run"
    [2018-02-08T14:31:21,731][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@use_column_value = false
    [2018-02-08T14:31:21,731][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@tracking_column_type = "numeric"
    [2018-02-08T14:31:21,745][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@clean_run = false
    [2018-02-08T14:31:21,746][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@record_last_run = true
    [2018-02-08T14:31:21,808][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@lowercase_column_names = true
    [2018-02-08T14:31:21,808][DEBUG][logstash.inputs.jdbc     ] config LogStash::Inputs::Jdbc/@columns_charset = {}
    [2018-02-08T14:31:21,830][DEBUG][logstash.plugins.registry] On demand adding plugin to the registry {:name=>"stdout", :type=>"output", :class=>LogStash::Outputs::Stdout}
    [2018-02-08T14:31:21,893][DEBUG][logstash.plugins.registry] On demand adding plugin to the registry {:name=>"json_lines", :type=>"codec", :class=>LogStash::Codecs::JSONLines}
    [2018-02-08T14:31:21,901][DEBUG][logstash.codecs.jsonlines] config LogStash::Codecs::JSONLines/@id = "json_lines_e27ae5ff-5352-4061-9415-c75234fafc91"
    [2018-02-08T14:31:21,902][DEBUG][logstash.codecs.jsonlines] config LogStash::Codecs::JSONLines/@enable_metric = true
    [2018-02-08T14:31:21,902][DEBUG][logstash.codecs.jsonlines] config LogStash::Codecs::JSONLines/@charset = "UTF-8"
    [2018-02-08T14:31:21,905][DEBUG][logstash.codecs.jsonlines] config LogStash::Codecs::JSONLines/@delimiter = "\n"
    [2018-02-08T14:31:21,915][DEBUG][logstash.outputs.stdout  ] config LogStash::Outputs::Stdout/@codec = <LogStash::Codecs::JSONLines id=>"json_lines_e27ae5ff-5352-4061-9415-c75234fafc91", enable_metric=>true, charset=>"UTF-8", delimiter=>"\n">
    [2018-02-08T14:31:21,924][DEBUG][logstash.outputs.stdout  ] config LogStash::Outputs::Stdout/@id = "4fb47c5631fa87c6a839a6f476077e9fa55456c479eee7251568f325435f3bbc"
    [2018-02-08T14:31:21,929][DEBUG][logstash.outputs.stdout  ] config LogStash::Outputs::Stdout/@enable_metric = true
    [2018-02-08T14:31:21,939][DEBUG][logstash.outputs.stdout  ] config LogStash::Outputs::Stdout/@workers = 1
    [2018-02-08T14:31:23,217][DEBUG][logstash.plugins.registry] On demand adding plugin to the registry {:name=>"amazon_es", :type=>"output", :class=>LogStash::Outputs::AmazonES}
    [2018-02-08T14:31:23,287][DEBUG][logstash.codecs.plain    ] config LogStash::Codecs::Plain/@id = "plain_673a059d-4236-4f10-ba64-43ee33e050e4"
    [2018-02-08T14:31:23,288][DEBUG][logstash.codecs.plain    ] config LogStash::Codecs::Plain/@enable_metric = true
    [2018-02-08T14:31:23,288][DEBUG][logstash.codecs.plain    ] config LogStash::Codecs::Plain/@charset = "UTF-8"
    [2018-02-08T14:31:23,294][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@hosts = ["search-XXXXXXXXXXXXXX.eu-west-3.es.amazonaws.com"]
    [2018-02-08T14:31:23,294][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@region = "eu-west-3"
    [2018-02-08T14:31:23,295][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@aws_access_key_id = "XXXXXXXXXXX"
    [2018-02-08T14:31:23,295][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@aws_secret_access_key = "XXXXXXXXXXXXX"
    [2018-02-08T14:31:23,296][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@index = "test-migrate"
    [2018-02-08T14:31:23,299][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@document_type = "data"
    [2018-02-08T14:31:23,299][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@id = "7c6401c2f72c63f8d359a42a2f440a663303cb2cbfefff8fa32d64a6f571a527"
    [2018-02-08T14:31:23,306][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@enable_metric = true
    [2018-02-08T14:31:23,310][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@codec = <LogStash::Codecs::Plain id=>"plain_673a059d-4236-4f10-ba64-43ee33e050e4", enable_metric=>true, charset=>"UTF-8">
    [2018-02-08T14:31:23,310][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@workers = 1
    [2018-02-08T14:31:23,310][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@manage_template = true
    [2018-02-08T14:31:23,317][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@template_name = "logstash"
    [2018-02-08T14:31:23,325][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@template_overwrite = false
    [2018-02-08T14:31:23,326][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@port = 443
    [2018-02-08T14:31:23,332][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@protocol = "https"
    [2018-02-08T14:31:23,333][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@flush_size = 500
    [2018-02-08T14:31:23,335][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@idle_flush_time = 1
    [2018-02-08T14:31:23,340][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@action = "index"
    [2018-02-08T14:31:23,341][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@path = "/"
    [2018-02-08T14:31:23,341][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@max_retries = 3
    [2018-02-08T14:31:23,341][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@retry_max_items = 5000
    [2018-02-08T14:31:23,342][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@retry_max_interval = 5
    [2018-02-08T14:31:23,342][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@doc_as_upsert = false
    [2018-02-08T14:31:23,342][DEBUG][logstash.outputs.amazones] config LogStash::Outputs::AmazonES/@upsert = ""
    [2018-02-08T14:31:23,426][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"

Val*_*Val 6

您应该尝试自己添加索引模板。将此ES 6.x 模板复制到您的本地文件系统上,然后将template设置添加到您的amazon_es输出中,它应该可以工作:

  amazon_es {
    hosts => ["search-xxxxx.eu-west-3.es.amazonaws.com"]
    region => "eu-west-3"
    aws_access_key_id => 'xxxxxxxxxxxxxxxxxxxxxx'
    aws_secret_access_key => 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
    index => "test-migrate"
    document_type => "data"
    template => '/path/to/template.json'
  }
Run Code Online (Sandbox Code Playgroud)