Google Cloud Container优化的操作系统主机记录到stackdriver

Question

TL; DR
将容器优化的os 主机日志(ssh和执行的shell命令)发送到Stackdriver 的最佳做法是什么？

背景:
我正在使用Googles Container Optimized OS,效果很好.将容器日志发送到Stackdriver非常容易,但如何将主机日志发送到Stackdriver？

这是出于审计目的,我需要记录所有SSH连接(接受或拒绝)以及通过shell执行的所有命令.以前我只是通过stackdriver主机记录器包将rsyslogd(auth,authpriv)发送到stackdriver.

这适用于在托管实例组(mig)中运行的Container Optimized OS VM,而不是在Google Kubernetes Engine中运行.

它可能非常明显,但我似乎无法找到任何文档.

Answer 1

从高层次来看，这就是任何 GCP COS 实例将操作系统审核日志发送到 Google stackdriver 所需执行的操作：

首先，您需要使用以下命令在 COS 上启用审核日志： systemctl start cloud-audit-setup 这将允许在计算实例日志中生成和捕获审核日志，您可以使用journalctl命令查看结果

其次，您需要在实例上安装 Google Stackdriver 代理并配置为将审核日志从实例日志传送到堆栈驱动程序。这可以通过让 docker 容器运行 Fluentd-gcp google 容器镜像来实现。

我正在分享下面的 cloud-init 来为您完成整个工作。您需要做的就是拥有一个带有“user-data”键的实例元数据，值是以下脚本：

#cloud-config
users:
- name: logger
  uid: 2001
  groups: docker

write_files:

- path: /etc/google-fluentd/fluentd.conf
  permissions: 0644
  owner: root
  content: |
    # This config comes from a heavily trimmed version of the
    # container-engine-customize-fluentd project. The upstream config is here:
    # https://github.com/GoogleCloudPlatform/container-engine-customize-fluentd/blob/6a46d72b29f3d8e8e495713bc3382ce28caf744e/kubernetes/fluentd- 
configmap.yaml
    <source>
        type systemd
        path /var/log/journal
        pos_file /var/log/gcp-journald.pos
        filters [{ "SYSLOG_IDENTIFIER": "audit" }]  
        tag node-journal
        read_from_head true
    </source>
    <match **>
      @type copy
       <store>
        @type google_cloud
        # Set the buffer type to file to improve the reliability
        # and reduce the memory consumption
        buffer_type file
        buffer_path /var/log/google-fluentd/cos-system.buffer
        # Set queue_full action to block because we want to pause gracefully
        # in case of the off-the-limits load instead of throwing an exception
        buffer_queue_full_action block
        # Set the chunk limit conservatively to avoid exceeding the GCL limit
        # of 10MiB per write request.
        buffer_chunk_limit 2M
        # Cap the combined memory usage of this buffer and the one below to
        # 2MiB/chunk * (6 + 2) chunks = 16 MiB
        buffer_queue_limit 6
        # Never wait more than 5 seconds before flushing logs in the non-error
        # case.
        flush_interval 5s
        # Never wait longer than 30 seconds between retries.
        max_retry_wait 30
        # Disable the limit on the number of retries (retry forever).
        disable_retry_limit
        # Use multiple threads for processing.
        num_threads 2
      </store>
    </match>
- path: /etc/systemd/system/logger.service
  permissions: 0644
  owner: root
  content: |
    [Unit]
    Description=logging docker container
    Requires=network-online.target
    After=network-online.target

    [Service]
    Environment="HOME=/home/logger"
    ExecStartPre=/usr/share/google/dockercfg_update.sh
    ExecStartPre=/bin/mkdir -p /var/log/google-fluentd/
    ExecStartPre=-/usr/bin/docker rm -fv logger
    ExecStart=/usr/bin/docker run --rm -u 0 \
       --name=logger \
       -v /var/log/:/var/log/ \
       -v /var/lib/docker/containers:/var/lib/docker/containers \
       -v /etc/google-fluentd/:/etc/fluent/config.d/ \
       --env='FLUENTD_ARGS=-q' \
       gcr.io/google-containers/fluentd-gcp:2.0.17
    Restart=always
    RestartSec=1
runcmd:
- systemctl daemon-reload
- systemctl start logger.service
- systemctl start cloud-audit-setup