多个入口控制器不工作

Question

我正在不同的命名空间中创建多个入口控制器。最初，它在 AWS 中创建一个负载均衡器，并将 Pod IP 地址附加到目标组。几天后，它不会将新的 Pod IP 更新到目标组。我已在此处附加入口控制器日志。

\n

E0712 15:02:30.516295       1 leaderelection.go:270] error retrieving resource lock namespace1/ingress-controller-leader-alb: configmaps "ingress-controller-le \xe2\x94\x82\n\xe2\x94\x82 ader-alb" is forbidden: User "system:serviceaccount:namespace1:fc-serviceaccount-icalb" cannot get resource "configmaps" in API group "" in the namespace "namespace1"\n

\n

入口.yaml

\n

apiVersion: extensions/v1beta1\nkind: Ingress\nmetadata:\n  name: "fc-ingress"\n  annotations:\n    kubernetes.io/ingress.class: alb-namespace1\n    alb.ingress.kubernetes.io/scheme: internet-facing\n    alb.ingress.kubernetes.io/subnets:\n    alb.ingress.kubernetes.io/certificate-arn: \n    alb.ingress.kubernetes.io/ssl-policy:\n    alb.ingress.kubernetes.io/security-groups:\n    alb.ingress.kubernetes.io/target-type: ip\n    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'\n    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP\n    alb.ingress.kubernetes.io/healthcheck-port: traffic-port\n    alb.ingress.kubernetes.io/healthcheck-path: '/'\n    alb.ingress.kubernetes.io/healthcheck-timeout-seconds:  '2'\n    alb.ingress.kubernetes.io/healthcheck-interval-seconds: '5'\n    alb.ingress.kubernetes.io/success-codes: '200'\n    alb.ingress.kubernetes.io/healthy-threshold-count: '5'\n    alb.ingress.kubernetes.io/unhealthy-threshold-count: '2'\n    alb.ingress.kubernetes.io/load-balancer-attributes: access_logs.s3.enabled=false\n    alb.ingress.kubernetes.io/load-balancer-attributes: deletion_protection.enabled=false\n    alb.ingress.kubernetes.io/load-balancer-attributes: routing.http2.enabled=true\n    alb.ingress.kubernetes.io/target-group-attributes: slow_start.duration_seconds=0\n    alb.ingress.kubernetes.io/target-group-attributes: deregistration_delay.timeout_seconds=300\n    alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=false\n  labels:\n    app: fc-label-app-ingress\nspec:\n  rules:\n     - host: "hostname1.com"\n       http:\n        paths:\n        - backend:\n            serviceName: service1\n            servicePort: 80\n\n     - host: "hostname2.com"\n       http:\n        paths:\n        - backend:\n            serviceName: service2\n            servicePort: 80\n\n     - host: "hostname3.com"\n       http:\n         paths:\n         - backend:\n             serviceName: service3\n             servicePort: 80\n\n\n

\n

ingress_controller.yaml

\n

# Application Load Balancer (ALB) Ingress Controller Deployment Manifest.\n# This manifest details sensible defaults for deploying an ALB Ingress Controller.\n# GitHub: https://github.com/kubernetes-sigs/aws-alb-ingress-controller\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  labels:\n    app: fc-label-app-icalb\n  name: fc-ingress-controller-alb\n  namespace: namespace1\n  # Namespace the ALB Ingress Controller should run in. Does not impact which\n  # namespaces it's able to resolve ingress resource for. For limiting ingress\n  # namespace scope, see --watch-namespace.\nspec:\n  replicas: 1\n  selector:\n    matchLabels:\n      app: fc-label-app-icalb\n  strategy:\n    rollingUpdate:\n      maxSurge: 1\n      maxUnavailable: 1\n    type: RollingUpdate\n  template:\n    metadata:\n      creationTimestamp: null\n      labels:\n        app: fc-label-app-icalb\n    spec:\n      containers:\n        - args:\n            # Limit the namespace where this ALB Ingress Controller deployment will\n            # resolve ingress resources. If left commented, all namespaces are used.\n            - --watch-namespace=namespace1\n\n            # Setting the ingress-class flag below ensures that only ingress resources with the\n            # annotation kubernetes.io/ingress.class: "alb" are respected by the controller. You may\n            # choose any class you'd like for this controller to respect.\n            - --ingress-class=alb-namespace1\n\n            # Name of your cluster. Used when naming resources created\n            # by the ALB Ingress Controller, providing distinction between\n            # clusters.\n            - --cluster-name=$EKS_CLUSTER_NAME\n\n          # AWS VPC ID this ingress controller will use to create AWS resources.\n            # If unspecified, it will be discovered from ec2metadata.\n            # - --aws-vpc-id=vpc-xxxxxx\n\n            # AWS region this ingress controller will operate in.\n            # If unspecified, it will be discovered from ec2metadata.\n            # List of regions: http://docs.aws.amazon.com/general/latest/gr/rande.html#vpc_region\n            # - --aws-region=us-west-1\n\n            # Enables logging on all outbound requests sent to the AWS API.\n            # If logging is desired, set to true.\n            # - ---aws-api-debug\n            # Maximum number of times to retry the aws calls.\n            # defaults to 10.\n            # - --aws-max-retries=10\n          env:\n            # AWS key id for authenticating with the AWS API.\n            # This is only here for examples. It's recommended you instead use\n            # a project like kube2iam for granting access.\n            #- name: AWS_ACCESS_KEY_ID\n            #  value: KEYVALUE\n\n            # AWS key secret for authenticating with the AWS API.\n            # This is only here for examples. It's recommended you instead use\n            # a project like kube2iam for granting access.\n            #- name: AWS_SECRET_ACCESS_KEY\n            #  value: SECRETVALUE\n          # Repository location of the ALB Ingress Controller.\n          image: docker.io/amazon/aws-alb-ingress-controller:v1.1.4\n          imagePullPolicy: Always\n          name: server\n          resources: {}\n          terminationMessagePath: /dev/termination-log\n      dnsPolicy: ClusterFirst\n      restartPolicy: Always\n      securityContext: {}\n      terminationGracePeriodSeconds: 30\n      serviceAccountName: fc-serviceaccount-icalb\n\n\n\n\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n  labels:\n    app: fc-label-app-icalb\n  name: fc-clusterrole-icalb\nrules:\n  - apiGroups:\n      - ""\n      - extensions\n    resources:\n      - configmaps\n      - endpoints\n      - events\n      - ingresses\n      - ingresses/status\n      - services\n    verbs:\n      - create\n      - get\n      - list\n      - update\n      - watch\n      - patch\n  - apiGroups:\n      - ""\n      - extensions\n    resources:\n      - nodes\n      - pods\n      - secrets\n      - services\n      - namespaces\n    verbs:\n      - get\n      - list\n      - watch\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n  labels:\n    app: fc-label-app-icalb\n  name: fc-clusterrolebinding-icalb\nroleRef:\n  apiGroup: rbac.authorization.k8s.io\n  kind: ClusterRole\n  name: fc-clusterrole-icalb\nsubjects:\n  - kind: ServiceAccount\n    name: fc-serviceaccount-icalb\n    namespace: namespace1\n---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n  labels:\n    app: fc-label-app-icalb\n  name: fc-serviceaccount-icalb\n  namespace: namespace1\n\n

\n

Answer 1

我在AKS上遇到过类似的问题。我有两个 Nginx Ingress 控制器：

1. 外部 nginx-ingress
2. 内部 nginx-ingress

一次只有一个人在工作，无论是内部的还是外部的。

在为每一个指定一个唯一的之后，election-id问题就得到了解决。

我使用以下 HELM 图表：

Repository    = "https://kubernetes.github.io/ingress-nginx"
Chart         = "ingress-nginx"
Chart_version = "4.1.3"
K8s Version   = "1.22.4"

部署

Repository    = "https://kubernetes.github.io/ingress-nginx"
Chart         = "ingress-nginx"
Chart_version = "4.1.3"
K8s Version   = "1.22.4"

入口类

kubectl get deploy -n ingress
NAME                                READY   UP-TO-DATE   AVAILABLE
external-nginx-ingress-controller   3/3     3            3           
internal-nginx-ingress-controller   1/1     1            1           

外部部署

kubectl get ingressclass
NAME             CONTROLLER                      PARAMETERS
external-nginx   k8s.io/ingress-nginx            <none>
internal-nginx   k8s.io/internal-ingress-nginx   <none>
   

内部部署

apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-nginx-ingress-controller
  namespace: ingress
  annotations:
    meta.helm.sh/release-name: external-nginx-ingress
    meta.helm.sh/release-namespace: ingress
spec:
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: external-nginx-ingress
      app.kubernetes.io/name: ingress-nginx
  template:
    spec:
      containers:
        - name: ingress-nginx-external-controller
          image: >-
            k8s.gcr.io/ingress-nginx/controller:v1.2.1
          args:
            - /nginx-ingress-controller
            - >-
              --publish-service=$(POD_NAMESPACE)/external-nginx-ingress-controller
            - '--election-id=external-ingress-controller-leader'
            - '--controller-class=k8s.io/ingress-nginx'
            - '--ingress-class=external-nginx'
            - '--ingress-class-by-name=true'