证书对 ingress.local 有效,对 gitlab.mydomain 无效

MHo*_*gge 5 gitlab kubernetes

我是 Kubernetes 新手。

  • 我使用 RKE(用于创建 k8 集群的 rancher 工具)安装了一个全新的 Kubernetes 集群。
  • 我添加了 gitlab 图表 ( https://charts.gitlab.io/ ) 并启动它。
  • 我设法解决了与持久存储等有关的几个问题。

但我现在陷入了最后一个问题:podgitlab-runner失败并显示以下日志:

ERROR: Registering runner... failed runner=Mk5hMxa5 status=couldn't execute POST against https://gitlab.mydomain.com/api/v4/runners: Post https://gitlab.mydomain.com/api/v4/runners: x509: certificate is valid for ingress.local, not gitlab.mydomain.com
PANIC: Failed to register this runner. Perhaps you are having network problems
Run Code Online (Sandbox Code Playgroud)

证书的描述使用kubectl describe certificate gitlab-gitlab-tls -n gitlab

Name:         gitlab-gitlab-tls
Namespace:    gitlab
Labels:       app=unicorn
              chart=unicorn-2.4.6
              heritage=Tiller
              io.cattle.field/appId=gitlab
              release=gitlab
Annotations:  <none>
API Version:  certmanager.k8s.io/v1alpha1
Kind:         Certificate
Metadata:
  Creation Timestamp:  2019-11-13T13:49:10Z
  Generation:          3
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  gitlab-unicorn
    UID:                   5640645f-550b-4073-bdf0-df8b089b0c94
  Resource Version:        6824
  Self Link:               /apis/certmanager.k8s.io/v1alpha1/namespaces/gitlab/certificates/gitlab-gitlab-tls
  UID:                     30ac32bd-c7f3-4f9b-9e3b-966b6090e1a9
Spec:
  Acme:
    Config:
      Domains:
        gitlab.mydomain.com
      http01:
        Ingress Class:  gitlab-nginx
  Dns Names:
    gitlab.mydomain.com
  Issuer Ref:
    Kind:       Issuer
    Name:       gitlab-issuer
  Secret Name:  gitlab-gitlab-tls
Status:
  Conditions:
    Last Transition Time:  2019-11-13T13:49:10Z
    Message:               Certificate issuance in progress. Temporary certificate issued.
    Reason:                TemporaryCertificate
    Status:                False
    Type:                  Ready
Events:                    <none>
Run Code Online (Sandbox Code Playgroud)

发行人的描述使用kubectl describe issuer gitlab-issuer -n gitlab

Name:         gitlab-issuer
Namespace:    gitlab
Labels:       app=certmanager-issuer
              chart=certmanager-issuer-0.1.0
              heritage=Tiller
              release=gitlab
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"certmanager.k8s.io/v1alpha1","kind":"Issuer","metadata":{"annotations":{},"creationTimestamp":"2019-11-13T13:49:10Z","gener...
API Version:  certmanager.k8s.io/v1alpha1
Kind:         Issuer
Metadata:
  Creation Timestamp:  2019-11-13T13:49:10Z
  Generation:          4
  Resource Version:    24537
  Self Link:           /apis/certmanager.k8s.io/v1alpha1/namespaces/gitlab/issuers/gitlab-issuer
  UID:                 b9971d7a-5220-47ca-a7f9-607aa3f9be4f
Spec:
  Acme:
    Email:  mh@mydomain.com
    http01:
    Private Key Secret Ref:
      Name:  gitlab-acme-key
    Server:  https://acme-v02.api.letsencrypt.org/directory
Status:
  Acme:
    Last Registered Email:  mh@mydomain.com
    Uri:                    https://acme-v02.api.letsencrypt.org/acme/acct/71695690
  Conditions:
    Last Transition Time:  2019-11-13T13:49:12Z
    Message:               The ACME account was registered with the ACME server
    Reason:                ACMEAccountRegistered
    Status:                True
    Type:                  Ready
Events:                    <none>
Run Code Online (Sandbox Code Playgroud)

使用的挑战描述kubectl describe challenges.certmanager.k8s.io -n gitlab gitlab-gitlab-tls-3386074437-0

Name:         gitlab-gitlab-tls-3386074437-0
Namespace:    gitlab
Labels:       acme.cert-manager.io/order-name=gitlab-gitlab-tls-3386074437
Annotations:  <none>
API Version:  certmanager.k8s.io/v1alpha1
Kind:         Challenge
Metadata:
  Creation Timestamp:  2019-11-13T13:49:15Z
  Finalizers:
    finalizer.acme.cert-manager.io
  Generation:  4
  Owner References:
    API Version:           certmanager.k8s.io/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Order
    Name:                  gitlab-gitlab-tls-3386074437
    UID:                   1f01771e-2e38-491f-9b2d-ab5f4fda60e2
  Resource Version:        6915
  Self Link:               /apis/certmanager.k8s.io/v1alpha1/namespaces/gitlab/challenges/gitlab-gitlab-tls-3386074437-0
  UID:                     4c115a6f-a76f-4859-a5db-6acd9c039d71
Spec:
  Authz URL:  https://acme-v02.api.letsencrypt.org/acme/authz-v3/1220588820
  Config:
    http01:
      Ingress Class:  gitlab-nginx
  Dns Name:           gitlab.mydomain.com
  Issuer Ref:
    Kind:    Issuer
    Name:    gitlab-issuer
  Key:       lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8.lPWns02SmS3zXwFzHdma_RyhwwlzWLRDkdlugFXDlZY
  Token:     lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8
  Type:      http-01
  URL:       https://acme-v02.api.letsencrypt.org/acme/chall-v3/1220588820/AwsnPw
  Wildcard:  false
Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for http-01 challenge propagation: wrong status code '404', expected '200'
  State:       pending
Events:        <none>
Run Code Online (Sandbox Code Playgroud)

Pod中发现的日志cert-manager

I1113 14:20:21.857235       1 pod.go:58] cert-manager/controller/challenges/http01/selfCheck/http01/ensurePod "level"=0 "msg"="found one existing HTTP01 solver pod" "dnsName"="gitlab.mydomain.com" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-ttkmj" "related_resource_namespace"="gitlab" "resource_kind"="Challenge" "resource_name"="gitlab-gitlab-tls-3386074437-0" "resource_namespace"="gitlab" "type"="http-01" 
I1113 14:20:21.857458       1 service.go:43] cert-manager/controller/challenges/http01/selfCheck/http01/ensureService "level"=0 "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="gitlab.mydomain.com" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-sdlw7" "related_resource_namespace"="gitlab" "resource_kind"="Challenge" "resource_name"="gitlab-gitlab-tls-3386074437-0" "resource_namespace"="gitlab" "type"="http-01" 
I1113 14:20:21.857592       1 ingress.go:91] cert-manager/controller/challenges/http01/selfCheck/http01/ensureIngress "level"=0 "msg"="found one existing HTTP01 solver ingress" "dnsName"="gitlab.mydomain.com" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-7jzwk" "related_resource_namespace"="gitlab" "resource_kind"="Challenge" "resource_name"="gitlab-gitlab-tls-3386074437-0" "resource_namespace"="gitlab" "type"="http-01" 
E1113 14:20:21.864785       1 sync.go:183] cert-manager/controller/challenges "msg"="propagation check failed" "error"="wrong status code '404', expected '200'" "dnsName"="gitlab.mydomain.com" "resource_kind"="Challenge" "resource_name"="gitlab-gitlab-tls-3386074437-0" "resource_namespace"="gitlab" "type"="http-01" 
Run Code Online (Sandbox Code Playgroud)
  • DNS gitlab.mydomain.com 设置为指向运行 NGINX 的 LoadBalancer 的 IP。
  • 如果我去https://gitlab.mydomain.com如果我在浏览器中
    • 浏览器显示连接不安全
    • 结果是“默认后端 - 404”。

编辑

使用以下方式描述入口控制器kubectl describe svc gitlab-nginx-ingress-controller -n gitlab

Name:              gitlab-nginx-ingress-controller
Namespace:         gitlab
Labels:            app=nginx-ingress
                   chart=nginx-ingress-0.30.0-1
                   component=controller
                   heritage=Tiller
                   io.cattle.field/appId=gitlab
                   release=gitlab
Annotations:       field.cattle.io/ipAddresses: null
                   field.cattle.io/targetDnsRecordIds: null
                   field.cattle.io/targetWorkloadIds: null
Selector:          <none>
Type:              ExternalName
IP:
External Name:     gitlab.mydomain.com
Port:              http  80/TCP
TargetPort:        http/TCP
NodePort:          http  31487/TCP
Endpoints:         10.42.0.7:80,10.42.1.9:80,10.42.2.12:80
Port:              https  443/TCP
TargetPort:        https/TCP
NodePort:          https  31560/TCP
Endpoints:         10.42.0.7:443,10.42.1.9:443,10.42.2.12:443
Port:              gitlab-shell  22/TCP
TargetPort:        gitlab-shell/TCP
NodePort:          gitlab-shell  30539/TCP
Endpoints:         10.42.0.7:22,10.42.1.9:22,10.42.2.12:22
Session Affinity:  None
Events:            <none>
Run Code Online (Sandbox Code Playgroud)

跑步kubectl get ingress -n gitlab给了我很多入口:

NAME                        HOSTS                 ADDRESS             PORTS     AGE
cm-acme-http-solver-5rjg4   minio.mydomain.com      gitlab.mydomain.com   80        4d23h
cm-acme-http-solver-7jzwk   gitlab.mydomain.com     gitlab.mydomain.com   80        4d23h
cm-acme-http-solver-tzs25   registry.mydomain.com   gitlab.mydomain.com   80        4d23h
gitlab-minio                minio.mydomain.com      gitlab.mydomain.com   80, 443   4d23h
gitlab-registry             registry.mydomain.com   gitlab.mydomain.com   80, 443   4d23h
gitlab-unicorn              gitlab.mydomain.com     gitlab.mydomain.com   80, 443   4d23h
Run Code Online (Sandbox Code Playgroud)

gitlab-unicorn使用说明kubectl describe ingress gitlab-unicron -n gitlab

Name:             gitlab-unicorn
Namespace:        gitlab
Address:          gitlab.mydomain.com
Default backend:  default-http-backend:80 (<none>)
TLS:
  gitlab-gitlab-tls terminates gitlab.mydomain.com
Rules:
  Host               Path  Backends
  ----               ----  --------
  gitlab.mydomain.com
                     /                gitlab-unicorn:8181 (10.42.0.9:8181,10.42.1.8:8181)
                     /admin/sidekiq   gitlab-unicorn:8080 (10.42.0.9:8080,10.42.1.8:8080)
Annotations:
  certmanager.k8s.io/issuer:                          gitlab-issuer
  field.cattle.io/publicEndpoints:                    [{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"gitlab:gitlab-unicorn","ingressName":"gitlab:gitlab-unicorn","hostname":"gitlab.mydomain.com","path":"/","allNodes":false},{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"gitlab:gitlab-unicorn","ingressName":"gitlab:gitlab-unicorn","hostname":"gitlab.mydomain.com","path":"/admin/sidekiq","allNodes":false}]
  kubernetes.io/ingress.class:                        gitlab-nginx
  kubernetes.io/ingress.provider:                     nginx
  nginx.ingress.kubernetes.io/proxy-body-size:        512m
  nginx.ingress.kubernetes.io/proxy-connect-timeout:  15
  nginx.ingress.kubernetes.io/proxy-read-timeout:     600
Events:                                               <none>
Run Code Online (Sandbox Code Playgroud)

cm-acme-http-solver-7jzwk使用说明kubectl describe ingress cm-acme-http-solver-7jzwk -n gitlab

Name:             cm-acme-http-solver-7jzwk
Namespace:        gitlab
Address:          gitlab.mydomain.com
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  gitlab.mydomain.com
                     /.well-known/acme-challenge/lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8   cm-acme-http-solver-sdlw7:8089 (10.42.2.19:8089)
Annotations:
  field.cattle.io/publicEndpoints:                     [{"addresses":[""],"port":80,"protocol":"HTTP","serviceName":"gitlab:cm-acme-http-solver-sdlw7","ingressName":"gitlab:cm-acme-http-solver-7jzwk","hostname":"gitlab.mydomain.com","path":"/.well-known/acme-challenge/lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8","allNodes":false}]
  kubernetes.io/ingress.class:                         gitlab-nginx
  nginx.ingress.kubernetes.io/whitelist-source-range:  0.0.0.0/0,::/0
Events:                                                <none>
Run Code Online (Sandbox Code Playgroud)

在我的 LoadBalancer 和集群的每个节点上打开端口(我知道我应该关闭一些端口,但我将首先设法使我的 gitlab 设置正常工作):

80/tcp                     ALLOW       Anywhere
443/tcp                    ALLOW       Anywhere
22/tcp                     ALLOW       Anywhere
2376/tcp                   ALLOW       Anywhere
2379/tcp                   ALLOW       Anywhere
2380/tcp                   ALLOW       Anywhere
6443/tcp                   ALLOW       Anywhere
6783/tcp                   ALLOW       Anywhere
6783:6784/udp              ALLOW       Anywhere
8472/udp                   ALLOW       Anywhere
4789/udp                   ALLOW       Anywhere
9099/tcp                   ALLOW       Anywhere
10250/tcp                  ALLOW       Anywhere
10254/tcp                  ALLOW       Anywhere
30000:32767/tcp            ALLOW       Anywhere
30000:32767/udp            ALLOW       Anywhere
80/tcp (v6)                ALLOW       Anywhere (v6)
443/tcp (v6)               ALLOW       Anywhere (v6)
22/tcp (v6)                ALLOW       Anywhere (v6)
2376/tcp (v6)              ALLOW       Anywhere (v6)
2379/tcp (v6)              ALLOW       Anywhere (v6)
2380/tcp (v6)              ALLOW       Anywhere (v6)
6443/tcp (v6)              ALLOW       Anywhere (v6)
6783/tcp (v6)              ALLOW       Anywhere (v6)
6783:6784/udp (v6)         ALLOW       Anywhere (v6)
8472/udp (v6)              ALLOW       Anywhere (v6)
4789/udp (v6)              ALLOW       Anywhere (v6)
9099/tcp (v6)              ALLOW       Anywhere (v6)
10250/tcp (v6)             ALLOW       Anywhere (v6)
10254/tcp (v6)             ALLOW       Anywhere (v6)
30000:32767/tcp (v6)       ALLOW       Anywhere (v6)
30000:32767/udp (v6)       ALLOW       Anywhere (v6)
Run Code Online (Sandbox Code Playgroud)

kubectl get pods -n gitlab

cm-acme-http-solver-4d8s5                               1/1     Running            0          5d
cm-acme-http-solver-ttkmj                               1/1     Running            0          5d
cm-acme-http-solver-ws7kv                               1/1     Running            0          5d
gitlab-certmanager-57bc6fb4fd-6rfds                     1/1     Running            0          5d
gitlab-gitaly-0                                         1/1     Running            0          5d
gitlab-gitlab-exporter-57b99467d4-knbgk                 1/1     Running            0          5d
gitlab-gitlab-runner-64b74bcd59-mxwvm                   0/1     CrashLoopBackOff   10         55m
gitlab-gitlab-shell-cff8b68f7-zng2c                     1/1     Running            0          5d
gitlab-gitlab-shell-cff8b68f7-zqvfr                     1/1     Running            0          5d
gitlab-issuer.1-lqs7c                                   0/1     Completed          0          5d
gitlab-migrations.1-c4njn                               0/1     Completed          0          5d
gitlab-minio-75567fcbb6-jjxhw                           1/1     Running            6          5d
gitlab-minio-create-buckets.1-6zljh                     0/1     Completed          0          5d
gitlab-nginx-ingress-controller-698fbc4c64-4wt97        1/1     Running            0          5d
gitlab-nginx-ingress-controller-698fbc4c64-5kv2h        1/1     Running            0          5d
gitlab-nginx-ingress-controller-698fbc4c64-jxljq        1/1     Running            0          5d
gitlab-nginx-ingress-default-backend-6cd54c5f86-2jrkd   1/1     Running            0          5d
gitlab-nginx-ingress-default-backend-6cd54c5f86-cxlmx   1/1     Running            0          5d
gitlab-postgresql-66d8d9574b-hbx78                      2/2     Running            0          5d
gitlab-prometheus-server-6fb685b9c7-c8bqj               2/2     Running            0          5d
gitlab-redis-7668c4d476-tcln5                           2/2     Running            0          5d
gitlab-registry-7bb984c765-7ww6j                        1/1     Running            0          5d
gitlab-registry-7bb984c765-t5jjq                        1/1     Running            0          5d
gitlab-sidekiq-all-in-1-8fd95bf7b-hfnjz                 1/1     Running            0          5d
gitlab-task-runner-5cd7bf5bb9-gnv8p                     1/1     Running            0          5d
gitlab-unicorn-864bd864f5-47zxg                         2/2     Running            0          5d
gitlab-unicorn-864bd864f5-gjms2                         2/2     Running            0          5d
Run Code Online (Sandbox Code Playgroud)

它们是 3 个 acme-http-solver:

  • 一份用于registry.mydomain.com
  • 一份用于 minio.mydomain.com
  • 一份用于 gitlab.mydomain.com

指向的日志gitlab.mydomain.com

I1113 13:49:21.207782       1 solver.go:39] cert-manager/acmesolver "level"=0 "msg"="starting listener"  "expected_domain"="gitlab.mydomain.com" "expected_key"="lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8.lPWns02SmS3zXwFzHdma_RyhwwlzWLRDkdlugFXDlZY" "expected_token"="lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8" "listen_port"=8089
Run Code Online (Sandbox Code Playgroud)

结果kubectl get svc -n gitlab

cm-acme-http-solver-48b2j                 NodePort       10.43.58.52     <none>              8089:30090/TCP                            5d23h
cm-acme-http-solver-h42mk                 NodePort       10.43.23.141    <none>              8089:30415/TCP                            5d23h
cm-acme-http-solver-sdlw7                 NodePort       10.43.86.27     <none>              8089:32309/TCP                            5d23h
gitlab-gitaly                             ClusterIP      None            <none>              8075/TCP,9236/TCP                         5d23h
gitlab-gitlab-exporter                    ClusterIP      10.43.187.247   <none>              9168/TCP                                  5d23h
gitlab-gitlab-shell                       ClusterIP      10.43.246.124   <none>              22/TCP                                    5d23h
gitlab-minio-svc                          ClusterIP      10.43.117.249   <none>              9000/TCP                                  5d23h
gitlab-nginx-ingress-controller           ExternalName   <none>          gitlab.mydomain.com   80:31487/TCP,443:31560/TCP,22:30539/TCP   5d23h
gitlab-nginx-ingress-controller-metrics   ClusterIP      10.43.152.252   <none>              9913/TCP                                  5d23h
gitlab-nginx-ingress-controller-stats     ClusterIP      10.43.173.191   <none>              18080/TCP                                 5d23h
gitlab-nginx-ingress-default-backend      ClusterIP      10.43.116.121   <none>              80/TCP                                    5d23h
gitlab-postgresql                         ClusterIP      10.43.97.139    <none>              5432/TCP                                  5d23h
gitlab-prometheus-server                  ClusterIP      10.43.67.220    <none>              80/TCP                                    5d23h
gitlab-redis                              ClusterIP      10.43.36.138    <none>              6379/TCP,9121/TCP                         5d23h
gitlab-registry                           ClusterIP      10.43.54.244    <none>              5000/TCP                                  5d23h
gitlab-unicorn                            ClusterIP      10.43.76.61     <none>              8080/TCP,8181/TCP                         5d23h
Run Code Online (Sandbox Code Playgroud)

Pod 的日志gitlab-nginx-ingress-controller-698fbc4c64-jxljq(其他 nginx-ingress-controller 提供相同的日志):https://textuploader.com/1o9we


关于我的配置可能有什么问题的任何提示吗?

欢迎询问有关我的设置的更多信息。

非常感谢。

Wes*_*ley 1

http01挑战依赖于公开的端口 80 (http) 才能回答挑战。该选项controller.service.enableHttp配置 http,默认情况下启用(请参阅此处。但即使您没有接触过此配置,也可能存在上游组件(即防火墙)阻止端口 80 上的流量。

您能否检查您的入口是否Service正在侦听端口 80,并且可以通过互联网访问?您可以尝试通过浏览器访问端口 80 上的公共 IP,检查是否收到来自入口控制器(或后端)的响应。