使用 Terraform 在 EKS 上部署 AWS 负载均衡器控制器

Question

尝试在 Kubernetes 上部署aws-load-balancer-controller。

我有以下 TF 代码：

resource "kubernetes_deployment" "ingress" {
  metadata {
    name      = "alb-ingress-controller"
    namespace = "kube-system"
    labels = {
      app.kubernetes.io/name = "alb-ingress-controller" 
      app.kubernetes.io/version = "v2.2.3"
      app.kubernetes.io/managed-by = "terraform"
    }
  }

  spec {
    replicas = 1

    selector {
      match_labels = {
        app.kubernetes.io/name = "alb-ingress-controller" 
      }
    }

    strategy {
      type = "Recreate"
    }

    template {
      metadata {
        labels = {
                app.kubernetes.io/name = "alb-ingress-controller" 
                app.kubernetes.io/version = "v2.2.3"
        }
      }

      spec {
        dns_policy                       = "ClusterFirst"
        restart_policy                   = "Always"
        service_account_name             = kubernetes_service_account.ingress.metadata[0].name
        termination_grace_period_seconds = 60

        container {
          name              = "alb-ingress-controller"
          image             = "docker.io/amazon/aws-alb-ingress-controller:v2.2.3"
          image_pull_policy = "Always"

          args = [
            "--ingress-class=alb",
            "--cluster-name=${local.k8s[var.env].esk_cluster_name}",
            "--aws-vpc-id=${local.k8s[var.env].cluster_vpc}",
            "--aws-region=${local.k8s[var.env].region}"
          ]
          volume_mount {
            mount_path = "/var/run/secrets/kubernetes.io/serviceaccount"
            name       = kubernetes_service_account.ingress.default_secret_name
            read_only  = true
          }
        }
        volume {
          name = kubernetes_service_account.ingress.default_secret_name

          secret {
            secret_name = kubernetes_service_account.ingress.default_secret_name
          }
        }
      }
    }
  }

  depends_on = [kubernetes_cluster_role_binding.ingress]
}

resource "kubernetes_ingress" "app" {
  metadata {
    name      = "owncloud-lb"
    namespace = "fargate-node"
    annotations = {
      "kubernetes.io/ingress.class"           = "alb"
      "alb.ingress.kubernetes.io/scheme"      = "internet-facing"
      "alb.ingress.kubernetes.io/target-type" = "ip"
    }
    labels = {
      "app" = "owncloud"
    }
  }

  spec {
    backend {
      service_name = "owncloud-service"
      service_port = 80
    }
    rule {
      http {
        path {
          path = "/"
          backend {
            service_name = "owncloud-service"
            service_port = 80
          }
        }
      }
    }
  }
  depends_on = [kubernetes_service.app]
}

这适用于1.9所需的版本。一旦我升级到版本，2.2.3pod 就无法更新，并且在 pod 上出现以下错误：{"level":"error","ts":1629207071.4385357,"logger":"setup","msg":"unable to create controller","controller":"TargetGroupBinding","error":"no matches for kind \"TargetGroupBinding\" in version \"elbv2.k8s.aws/v1beta1\""}

我已阅读更新文档并修改了 IAM 政策，正如他们所说，但他们还提到：

更新 TargetGroupBinding CRD

我不确定如何使用 terraform 做到这一点

如果我尝试在新集群上进行部署（例如，不是从 1.9 升级，我会得到相同的错误），我也会得到相同的错误。

Answer 1

使用 Terraform 代码，您可以应用Deployment和资源，但还必须为自定义资源Ingress添加。CustomResourceDefinitionsTargetGroupBinding

负载均衡器控制器安装文档中的“将控制器添加到集群”中对此进行了描述- 并提供了 Helm 和 Kubernetes Yaml 的示例。

Terraform 对应用 CRD 提供测试版支持，包括部署 CustomResourceDefinition 的示例。