Nginx ingress 忽略 ConfigMap 和注释

Question

我已经设置了一个k8s集群（目前1个裸机节点，既是master又是worker）。我还按照此处所述设置了 Nginx 入口控制器： https: //docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/以下是具体步骤：

1. kubectl apply -f common/ns-and-sa.yaml https://github.com/nginxinc/kubernetes-ingress/blob/release-1.11/deployments/common/ns-and-sa.yaml（无修改）
2. kubectl apply -f rbac/rbac.yaml https://github.com/nginxinc/kubernetes-ingress/blob/release-1.11/deployments/rbac/rbac.yaml（无修改）
3. kubectl apply -f common/default-server-secret.yaml https://github.com/nginxinc/kubernetes-ingress/blob/release-1.11/deployments/common/default-server-secret.yaml（无修改）
4. kubectl apply -f common/nginx-config.yaml https://github.com/nginxinc/kubernetes-ingress/blob/release-1.11/deployments/common/nginx-config.yaml修改文件：

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:
  ignore-invalid-headers: "false"
  use-forwarded-headers: "true"
  forwarded-for-header: "CF-Connecting-IP"
  proxy-real-ip-cidr: "...IPs go here..."

5. kubectl apply -f common/ingress-class.yaml https://github.com/nginxinc/kubernetes-ingress/blob/release-1.11/deployments/common/ingress-class.yaml修改后的文件：

apiVersion: networking.k8s.io/v1beta1
kind: IngressClass
metadata:
  name: nginx
  annotations:
    ingressclass.kubernetes.io/is-default-class: "true"
spec:
  controller: nginx.org/ingress-controller

6. 这些命令：

kubectl apply -f common/crds/k8s.nginx.org_virtualservers.yaml
kubectl apply -f common/crds/k8s.nginx.org_virtualserverroutes.yaml
kubectl apply -f common/crds/k8s.nginx.org_transportservers.yaml
kubectl apply -f common/crds/k8s.nginx.org_policies.yaml

没有修改，链接：

• https://github.com/nginxinc/kubernetes-ingress/blob/release-1.11/deployments/common/crds/k8s.nginx.org_virtualservers.yaml
• https://github.com/nginxinc/kubernetes-ingress/blob/release-1.11/deployments/common/crds/k8s.nginx.org_virtualserverroutes.yaml
• https://github.com/nginxinc/kubernetes-ingress/blob/release-1.11/deployments/common/crds/k8s.nginx.org_transportservers.yaml
• https://github.com/nginxinc/kubernetes-ingress/blob/release-1.11/deployments/common/crds/k8s.nginx.org_policies.yaml

7. kubectl apply -f daemon-set/nginx-ingress.yaml https://github.com/nginxinc/kubernetes-ingress/blob/release-1.11/deployments/daemon-set/nginx-ingress.yaml（无修改）

我还设置了证书管理器，它运行良好（很确定这并不重要）。

现在，当我创建一些 Ingress 资源时，它几乎可以工作了。我可以从外部互联网访问它，证书颁发工作等。但是没有应用ConfigMap（common/nginx-config.yaml），并且nginx.org/rewrite-target: /$1也没有应用类似的注释。

例子：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-com
  namespace: example-com
  annotations:
    nginx.org/rewrite-target: /$1
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - example.com
    secretName: example-com-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /api/(.*)
        pathType: ImplementationSpecific
        backend:
          service:
            name: api
            port:
              number: 80
      - path: /(.*)
        pathType: ImplementationSpecific
        backend:
          service:
            name: frontend
            port:
              number: 80

当然，使用真实域名。在此示例中，我收到 404 nginx 错误。在其他入口中，我传递/proxy-body-size注释，这也不起作用（无法上传大文件）。

我已经exec进入入口控制器容器并kubectl -n nginx-ingress exec -it nginx-ingress-snjjp bash查看了/etc/nginx/conf.d. 所有文件均不包含 ConfigMap 或注释中指定的配置。

这就是它的样子（我删除了多余的空行并替换了域名）：

# configuration for example-com/example-com

upstream example-com-example-com-example.com-api-80 {
        zone example-com-example-com-example.com-api-80 256k;
        random two least_conn;

        server 10.32.0.4:80 max_fails=1 fail_timeout=10s max_conns=0;
}
upstream example-com-example-com-example.com-frontend-80 {
        zone example-com-example-com-example.com-frontend-80 256k;
        random two least_conn;

        server 10.32.0.27:80 max_fails=1 fail_timeout=10s max_conns=0;
}

server {
        listen 80;

        listen 443 ssl;

        ssl_certificate /etc/nginx/secrets/example-com-example-com-tls;
        ssl_certificate_key /etc/nginx/secrets/example-com-example-com-tls;

        server_tokens on;

        server_name example.com;

        set $resource_type "ingress";
        set $resource_name "example-com";
        set $resource_namespace "example-com";

        if ($scheme = http) {
                return 301 https://$host:443$request_uri;
        }

        location /api/(.*) {
                set $service "api";

                proxy_http_version 1.1;

                proxy_connect_timeout 60s;
                proxy_read_timeout 60s;
                proxy_send_timeout 60s;
                client_max_body_size 1m;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Host $host;
                proxy_set_header X-Forwarded-Port $server_port;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_buffering on;

                proxy_pass http://example-com-example-com-example.com-api-80;
        }
        location /(.*) {
                set $service "frontend";

                proxy_http_version 1.1;

                proxy_connect_timeout 60s;
                proxy_read_timeout 60s;
                proxy_send_timeout 60s;
                client_max_body_size 1m;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Host $host;
                proxy_set_header X-Forwarded-Port $server_port;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_buffering on;

                proxy_pass http://example-com-example-com-example.com-frontend-80;
        }
}

我还尝试了nginx.ingress.kubernetes.io/注释（如你所见，我不是专业人士，这是我在谷歌上搜索的）。没有成功。

我正在更新我的集群，并且使用旧版本的 k8s（我认为是 1.15），几天前一切正常。当然，除了入口控制器之外，我对每个服务都使用了完全相同的配置。

有任何想法吗？

Answer 1

我已经发现出了什么问题。我在旧设置中使用 Kubernetes Nginx 入口控制器https://kubernetes.github.io/ingress-nginx/，现在我使用 Nginx 入口控制器https://www.nginx.com/products/nginx-ingress- controller/这些实现有不同的注解（后者缺少很多有用的注解）。这确实非常令人困惑，因为配置相似，人们可能会认为它们是相同的。