运行 terraform apply 会导致 Kubernetes 提供商在 Azure 云中部署 k8s 资源未经授权

Question

我正在使用AKSterraform 部署 k8s 集群。

群集已启用带有 azure Active Directory 的 rbac。

集群创建进展顺利，之后 terraform 尝试在集群上执行一些任务，例如创建k8s-roles storage classes....，并失败并显示Unauthorized错误消息，如下所示：

module.k8s_cluster.module.infra.kubernetes_storage_class.managed-premium-retain: Creating...
module.k8s_cluster.module.infra.kubernetes_cluster_role.containerlogs: Creating...
module.k8s_cluster.module.infra.kubernetes_namespace.add_pod_identity: Creating...
module.k8s_cluster.module.infra.kubernetes_storage_class.managed-standard-retain: Creating...
module.k8s_cluster.module.infra.kubernetes_storage_class.managed-premium-delete: Creating...
module.k8s_cluster.module.appgw.kubernetes_namespace.agic[0]: Creating...
module.k8s_cluster.module.infra.kubernetes_storage_class.managed-standard-delete: Creating...

Error: Unauthorized

  on .terraform/modules/k8s_cluster/modules/infra/k8s-roles.tf line 1, in resource "kubernetes_cluster_role" "containerlogs":
   1: resource "kubernetes_cluster_role" "containerlogs" {



Error: Unauthorized

  on .terraform/modules/k8s_cluster/modules/infra/k8s-storages-classes.tf line 1, in resource "kubernetes_storage_class" "managed-standard-retain":
   1: resource "kubernetes_storage_class" "managed-standard-retain" {



Error: Unauthorized

  on .terraform/modules/k8s_cluster/modules/infra/k8s-storages-classes.tf line 14, in resource "kubernetes_storage_class" "managed-standard-delete":
  14: resource "kubernetes_storage_class" "managed-standard-delete" {



Error: Unauthorized

  on .terraform/modules/k8s_cluster/modules/infra/k8s-storages-classes.tf line 27, in resource "kubernetes_storage_class" "managed-premium-retain":
  27: resource "kubernetes_storage_class" "managed-premium-retain" {



Error: Unauthorized

  on .terraform/modules/k8s_cluster/modules/infra/k8s-storages-classes.tf line 40, in resource "kubernetes_storage_class" "managed-premium-delete":
  40: resource "kubernetes_storage_class" "managed-premium-delete" {



Error: Unauthorized

  on .terraform/modules/k8s_cluster/modules/infra/r-aad-pod-identity.tf line 5, in resource "kubernetes_namespace" "add_pod_identity":
   5: resource "kubernetes_namespace" "add_pod_identity" {



Error: Unauthorized

  on .terraform/modules/k8s_cluster/modules/tools/agic/helm-agic.tf line 1, in resource "kubernetes_namespace" "agic":
   1: resource "kubernetes_namespace" "agic" {

如您所见，这些不是azure错误，而是kubernetes

我似乎无权在新创建的集群上执行上述资源创建任务。为了授予我的用户帐户执行这些 terraform 任务的权限，应该做什么以及在哪里做什么？

Answer 1

最简单的答案是更改您的 kubernetes 提供程序配置

provider "kubernetes" {
  load_config_file       = "false"
  host                   = azurerm_kubernetes_cluster.main.kube_config.0.host
  username               = azurerm_kubernetes_cluster.main.kube_config.0.username
  password               = azurerm_kubernetes_cluster.main.kube_config.0.password
  client_certificate     = "${base64decode(azurerm_kubernetes_cluster.main.kube_config.0.client_certificate)}"
  client_key             = "${base64decode(azurerm_kubernetes_cluster.main.kube_config.0.client_key)}"
  cluster_ca_certificate = "${base64decode(azurerm_kubernetes_cluster.main.kube_config.0.cluster_ca_certificate)}"
}

到

provider "kubernetes" {
  load_config_file       = "false"
  host                   = azurerm_kubernetes_cluster.main.kube_admin_config.0.host
  username               = azurerm_kubernetes_cluster.main.kube_admin_config.0.username
  password               = azurerm_kubernetes_cluster.main.kube_admin_config.0.password
  client_certificate     = "${base64decode(azurerm_kubernetes_cluster.main.kube_admin_config.0.client_certificate)}"
  client_key             = "${base64decode(azurerm_kubernetes_cluster.main.kube_admin_config.0.client_key)}"
  cluster_ca_certificate = "${base64decode(azurerm_kubernetes_cluster.main.kube_admin_config.0.cluster_ca_certificate)}"
}

如果本地帐户被禁用，这将不起作用，但您可以使用它：

provider "kubernetes" {
  host                   = data.azurerm_kubernetes_cluster.this.kube_config.0.host
  cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.this.kube_config.0.cluster_ca_certificate)
  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command = "./kubelogin"
    args = [
      "get-token",
      "--login",
      "spn",
      "--environment",
      "AzurePublicCloud",
      "--tenant-id",
      var.tenant_id,
      "--server-id",
      var.aad_server_id,
      "--client-id",
      var.client_id,
      "--client-secret",
      var.client_secret
    ]
  }
}

请注意，您需要在存储库中包含kubelogin二进制文件。更多详细信息请参见此处。