Serilog HTTP 接收器 + Logstash：将 Serilog 消息数组拆分为单个日志事件

Question

我们使用Serilog HTTP 接收器将消息发送到 Logstash。但是HTTP消息体是这样的：

{
  "events": [
    {
      "Timestamp": "2016-11-03T00:09:11.4899425+01:00",
      "Level": "Debug",
      "MessageTemplate": "Logging {@Heartbeat} from {Computer}",
      "RenderedMessage": "Logging { UserName: \"Mike\", UserDomainName: \"Home\" } from \"Workstation\"",
      "Properties": {
        "Heartbeat": {
          "UserName": "Mike",
          "UserDomainName": "Home"
        },
        "Computer": "Workstation"
      }
    },
    {
      "Timestamp": "2016-11-03T00:09:12.4905685+01:00",
      "Level": "Debug",
      "MessageTemplate": "Logging {@Heartbeat} from {Computer}",
      "RenderedMessage": "Logging { UserName: \"Mike\", UserDomainName: \"Home\" } from \"Workstation\"",
      "Properties": {
        "Heartbeat": {
          "UserName": "Mike",
          "UserDomainName": "Home"
        },
        "Computer": "Workstation"
      }
    }
  ]
}

IE。日志事件在一个数组中批处理。可以一条一条地发送消息，但它仍然是一个单项数组。

然后该事件在 Kibana 中显示为具有message带值的字段

{
  "events": [
    {
      // ...
    },
    {
      // ...
    }
  ]
}

IE。字面意思是来自 HTTP 输入的内容。

如何将events数组中的项目拆分为单独的日志记录事件并将属性“上拉”到顶层，以便在 ElasticSearch 中有两个日志记录事件：

---

  "Timestamp": "2016-11-03T00:09:11.4899425+01:00",
  "Level": "Debug",
  "MessageTemplate": "Logging {@Heartbeat} from {Computer}",
  "RenderedMessage": "Logging { UserName: \"Mike\", UserDomainName: \"Home\" } from \"Workstation\"",
  "Properties": {
    "Heartbeat": {
      "UserName": "Mike",
      "UserDomainName": "Home"
    },
    "Computer": "Workstation"
  }

---

  "Timestamp": "2016-11-03T00:09:12.4905685+01:00",
  "Level": "Debug",
  "MessageTemplate": "Logging {@Heartbeat} from {Computer}",
  "RenderedMessage": "Logging { UserName: \"Mike\", UserDomainName: \"Home\" } from \"Workstation\"",
  "Properties": {
    "Heartbeat": {
      "UserName": "Mike",
      "UserDomainName": "Home"
    },
    "Computer": "Workstation"
  }

---

我试过 Logstash json和split，但我不能让它工作。

Answer 1

升级到 Logstash 5.0 后，由于事件 API的更改， Val 的解决方案停止工作：更新未反映在原始版本中。对于 Logstash 5.0+ ，必须使用访问器。event.to_hasheventevent.get('field')event.set('field', value)

现在更新的解决方案是：

input {
  http {
    port => 8080
    codec => json
  }
}

filter {
  split {
    field => "events"
  }
  ruby {
    code => "
      event.get('events').each do |k, v|
        event.set(k, v)
      end
    "
  }
  mutate {
    remove_field => [ "events" ]
  }
}