Filebeat Kubernetes 处理器和过滤

Question

我正在尝试使用 Filebeat 将我的 K8s pod 日志发送到 Elasticsearch。

我在这里在线遵循指南：https : //www.elastic.co/guide/en/beats/filebeat/6.0/running-on-kubernetes.html

一切都按预期工作，但是我想从系统 pod 中过滤掉事件。我更新的配置看起来像：

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-prospectors
  namespace: kube-system
  labels:
    k8s-app: filebeat
    kubernetes.io/cluster-service: "true"
data:
  kubernetes.yml: |-
    - type: log
      paths:
        - /var/lib/docker/containers/*/*.log
  multiline.pattern: '^\s'
  multiline.match: after
  json.message_key: log
  json.keys_under_root: true
  processors:
    - add_kubernetes_metadata:
        in_cluster: true
        namespace: ${POD_NAMESPACE}
    - drop_event.when.regexp:
        or:
          kubernetes.pod.name: "weave-net.*"
          kubernetes.pod.name: "external-dns.*"
          kubernetes.pod.name: "nginx-ingress-controller.*"
          kubernetes.pod.name: "filebeat.*"

我试图忽视weave-net，external-dns，ingress-controller并filebeat通过事件：

- drop_event.when.regexp:
    or:
      kubernetes.pod.name: "weave-net.*"
      kubernetes.pod.name: "external-dns.*"
      kubernetes.pod.name: "nginx-ingress-controller.*"
      kubernetes.pod.name: "filebeat.*"

然而，它们继续到达 Elasticsearch。

Answer 1

条件需要是一个列表：

- drop_event.when.regexp:
    or:
      - kubernetes.pod.name: "weave-net.*"
      - kubernetes.pod.name: "external-dns.*"
      - kubernetes.pod.name: "nginx-ingress-controller.*"
      - kubernetes.pod.name: "filebeat.*"

我不确定您的参数顺序是否有效。我的一个工作示例如下所示：

- drop_event:
    when:
      or:
        # Exclude traces from Zipkin
        - contains.path: "/api/v"
        # Exclude Jolokia calls
        - contains.path: "/jolokia/?"
        # Exclude pinging metrics
        - equals.path: "/metrics"
        # Exclude pinging health
        - equals.path: "/health"

Answer 2

这在 filebeat 6.1.3 中对我有用

        - drop_event.when:
            or:
            - equals:
                kubernetes.container.name: "filebeat"
            - equals:
                kubernetes.container.name: "prometheus-kube-state-metrics"
            - equals:
                kubernetes.container.name: "weave-npc"
            - equals:
                kubernetes.container.name: "nginx-ingress-controller"
            - equals:
                kubernetes.container.name: "weave"