Prometheus：监控所有服务而不为每个服务创建ServiceMonitor？

Question

我正在使用这个 prometheus helm chart。

我想知道是否可以设置 prometheus 操作符来自动监控集群或命名空间中的每个服务，而不必ServiceMonitor为每个服务创建一个。

使用当前设置，当我想监视服务时，我必须创建一个ServiceMonitor带有标签release: prometheus.

编辑：

带monitoring: "true"标签的服务

apiVersion: v1
kind: Service
metadata:
  name: issue-manager-service
  labels:
    app: issue-manager-app
    monitoring: "true"
spec:
  selector:
    app: issue-manager-app
  ports:
    - protocol: TCP
      name: http
      port: 80
      targetPort: 7200

“包罗万象”的服务监视器：

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: service-monitor-scraper
  labels:
    release: prometheus
spec:
  endpoints:
  - port: metrics
    interval: 30s
    path: /metrics
  jobLabel: monitoring
  namespaceSelector:
    any: true
  selector:
    matchLabels:
      monitoring: "true"

Answer 1

仅当您在所有服务上都有一个共同的标签时

# for example:
org: "my-company"
# or
monitoring: "true"
# or 
app.kubernetes.io/managed-by: "Helm"  # <- in most cases this represents all

然后你定义一个单一的、跨命名空间ServiceMonitor，它涵盖了所有标记的服务：

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: common-monitor
  namespace: monitoring
spec:
  endpoints:
  - port: metrics
    interval: 30s
    path: /metrics
  jobLabel: monitoring
  namespaceSelector:
    any: true  # <- important
  selector:
    matchLabels:
      monitoring: "true"  # <- should match what you've chosen as common

然后，为了确保ServiceMonitorPrometheus Operator 发现了这一点，您可以：

• 此声明ServiceMonitor：通过内置的操作模板https://github.com/prometheus-community/helm-charts/blob/4164ad5fdb6a977f1aba7b65f4e65582d3081528/charts/kube-prometheus-stack/values.yaml#L2008
• 或者，声明一个serviceMonitorSelector指向你的ServiceMonitor https://github.com/prometheus-community/helm-charts/blob/4164ad5fdb6a977f1aba7b65f4e65582d3081528/charts/kube-prometheus-stack/values.yaml#L1760

Prometheus Operator 和 ServiceMonitor 之间的这种额外显式链接是有意完成的 - 这样，如果您的集群上有 2 个 Prometheus 实例（例如 Infra 和 Product），您可以将哪个 Prometheus 将哪些 Pod 分配到其抓取配置中。

从您的问题来看，听起来您已经有了一个serviceMonitorSelector基于release: prometheus标签的标签 - 尝试将其添加到您的全能标签中ServiceMonitor。