如何将运行在GCP之上的Kubernetes上的pod中的日志发送到elasticsearch/logstash？

Question

我在Google-Container-Engine中运行我系统的新模块.我想将stdout和stderr从他们(在pods中运行)带到我的集中式logstash.有没有一种简单的方法可以将日志从pod转发到外部日志服务,例如logstash或elasticsearch？

Answer 1

我决定直接登录elasticsearch，该外部虚拟机可以在elasticsearch.c.my-project.internal（我在Google-Cloud-Platform上）进行访问。这很容易：

1. 设置一个名称为elasticsearch的ExternalService，它指向elasticsearch实例：

   apiVersion: v1 kind: Service metadata: name: elasticsearch-logging namespace: kube-system labels: k8s-app: elasticsearch kubernetes.io/name: "elasticsearch" spec: type: ExternalName externalName: elasticsearch.c.my-project.internal ports: - port: 9200 targetPort: 9200

2. 将流利弹性搜索部署为DeamonSet。fluentd-elasticsearch将自动连接到具有名称的服务elasticsearch-logging（基于fluentd-elasticsearch部署定义：

   apiVersion: extensions/v1beta1 kind: DaemonSet metadata: name: fluentd-elasticsearch namespace: kube-system labels: tier: monitoring app: fluentd-logging k8s-app: fluentd-logging spec: template: metadata: labels: name: fluentd-elasticsearch spec: containers: - name: fluentd-elasticsearch image: gcr.io/google_containers/fluentd-elasticsearch:1.19 volumeMounts: - name: varlog mountPath: /var/log - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true terminationGracePeriodSeconds: 30 volumes: - name: varlog hostPath: path: /var/log - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers

   使用kubectl logs fluentd-elasticsearch-...来检查你是否能够连接到elasticsearach实例。

3. 现在，您可以访问kibana并查看日志。