Helm Prometheus 操作员不会向目标添加新的 ServiceMonitor 端点

Question

我正在尝试使用 helm prometheus https://github.com/prometheus-community/helm-charts监控我的应用程序。我已经成功安装了这个舵图。

prometheus-kube-prometheus-operator-5d8dcd5988-bw222   1/1     Running   0          11h
prometheus-kube-state-metrics-5d45f64d67-97vxt         1/1     Running   0          11h
prometheus-prometheus-kube-prometheus-prometheus-0     2/2     Running   0          11h
prometheus-prometheus-node-exporter-gl4cz              1/1     Running   0          11h
prometheus-prometheus-node-exporter-mxrsm              1/1     Running   0          11h
prometheus-prometheus-node-exporter-twvdb              1/1     Running   0          11h

通过这些 yml 配置在同一命名空间中创建应用服务和部署：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: appservice
  namespace: monitoring
  labels:
    app: appservice
  annotations:
    prometheus.io/scrape: 'true'
    prometheus.io/path: '/actuator/prometheus'
spec:
  replicas: 1
  selector:
    matchLabels:
      app: appservice
  template:
    metadata:
      labels:
        app: appservice
...

apiVersion: v1
kind: Service
metadata:
  name: appservice
  namespace: monitoring
  annotations:
    prometheus.io/scrape: 'true'
    prometheus.io/path: '/actuator/prometheus'
spec:
  selector:
    app: appservice
  type: ClusterIP
  ports:
    - name: web
      protocol: TCP
      port: 8080
      targetPort: 8080
    - name: jvm-debug
      protocol: TCP
      port: 5005
      targetPort: 5005

部署应用程序后，我创建了 ServiceMonitor：

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: appservice-servicemonitor
  namespace: monitoring
  labels:
    app: appservice
    release: prometheus-repo
spec:
  selector:
    matchLabels:
      app: appservice # target app service
  namespaceSelector:
    matchNames:
      - monitoring
  endpoints:
  - port: web
    path: '/actuator/prometheus'
    interval: 15s

我希望添加此 ServiceMonitor 后，我的 prometheus 实例会创建新的目标```，例如“http://appservice:8080/actuator/prometheus”，但事实并非如此，新端点不会出现在 prometheus UI 中。

我尝试通过添加 extraServiceMonitors 来更改 helm 值

namespaceOverride: "monitoring"
nodeExporter:
  enabled: true

prometheus:
  enabled: true
  prometheusSpec:
    serviceMonitorSelectorNilUsesHelmValues: false
    serviceMonitorSelector:
      matchLabels:
       release: prometheus-repo
    additionalServiceMonitors:
      namespaceSelector:
        any: true
    replicas: 1
    shards: 1
    storageSpec:
      ...
    securityContext:
      ...
    nodeSelector:
      assignment: monitoring

  nodeSelector:
    assignment: monitoring

prometheusOperator:
  nodeSelector:
    assignment: monitoring
  admissionWebhooks:
    patch:
      securityContext:
        ...
  securityContext:
    ...

global:
  alertmanagerSpec:
    nodeSelector:
      assignment: monitoring

但这没有帮助。真的很难说出了什么问题，没有错误日志，所有配置都成功应用。

Answer 1

我发现本指南非常有帮助。

请记住，根据您使用的 Prometheus 堆栈，标签和名称可以有不同的默认值（对我来说，使用 kube-prometheus-stack，例如秘密名称是 prometheus-kube-prometheus-stack-prometheus 而不是 prometheus -k8s）。

基本报价：

我的 ServiceMonitor 被 Prometheus 发现了吗？

ServiceMonitor 对象及其所属的命名空间由 Prometheus 对象的 serviceMonitorSelector 和 serviceMonitorNamespaceSelector 选择。ServiceMonitor 的名称在 Prometheus 配置中进行编码，因此您可以简单地 grep 它是否存在。Prometheus Operator 生成的配置存储在 Kubernetes Secret 中，该 Secret 以 Prometheus 对象名称命名，前缀为 prometheus-，并且与 Prometheus 对象位于同一命名空间中。例如，对于名为 k8s 的 Prometheus 对象，可以通过以下方式查明是否已选取名为 my-service-monitor 的 ServiceMonitor：

kubectl -n monitoring get secret prometheus-k8s -ojson | jq -r '.data["prometheus.yaml.gz"]' | base64 -d | gunzip | grep "my-service-monitor