GCP GKE：服务帐户“正在尝试授予当前未持有的 RBAC 权限”

Question

我正在设置 CI/CD 管道，用于以自动化方式部署基于 Kubernetes 的应用程序。此部署的一部分涉及创建其他服务帐户及其关联角色。

\n

当我的管道运行时，部署失败并显示以下错误消息：

\n

Error: roles.rbac.authorization.k8s.io "mongodb-kubernetes-operator" is forbidden: user "cicd-bot@my-project.iam.gserviceaccount.com" (groups=["system:authenticated"]) is attempting to grant RBAC permissions not currently held:\n\xe2\x94\x82 {APIGroups:[""], Resources:["configmaps"], Verbs:["list" "watch" "create" "update" "patch" "get" "delete"]}\n\xe2\x94\x82 {APIGroups:[""], Resources:["pods"], Verbs:["list" "watch" "create" "update" "patch" "get" "delete"]}\n\xe2\x94\x82 {APIGroups:[""], Resources:["secrets"], Verbs:["list" "watch" "create" "update" "patch" "get" "delete"]}\n\xe2\x94\x82 {APIGroups:[""], Resources:["services"], Verbs:["list" "watch" "create" "update" "patch" "get" "delete"]}\n\xe2\x94\x82 {APIGroups:["apps"], Resources:["statefulsets"], Verbs:["list" "watch" "create" "update" "patch" "get" "delete"]}\n\xe2\x94\x82 {APIGroups:["mongodbcommunity.mongodb.com"], Resources:["mongodbcommunity"], Verbs:["list" "watch" "update" "patch" "get"]}\n\xe2\x94\x82 {APIGroups:["mongodbcommunity.mongodb.com"], Resources:["mongodbcommunity/finalizers"], Verbs:["list" "watch" "update" "patch" "get"]}\n\xe2\x94\x82 {APIGroups:["mongodbcommunity.mongodb.com"], Resources:["mongodbcommunity/spec"], Verbs:["list" "watch" "update" "patch" "get"]}\n\xe2\x94\x82 {APIGroups:["mongodbcommunity.mongodb.com"], Resources:["mongodbcommunity/status"], Verbs:["list" "watch" "update" "patch" "get"]}\n\xe2\x94\x82 \n\xe2\x94\x82   with module.db_document.kubernetes_role.operator_mongodb,\n\xe2\x94\x82   on modules/db_document/main.tf line 17, in resource "kubernetes_role" "operator_mongodb":\n\xe2\x94\x82   17: resource "kubernetes_role" "operator_mongodb" {\n\xe2\x94\x82 \n

\n

该错误似乎很简单：我的服务帐户无法授予它没有的权限。由于错误消息提到了我的 GCP 服务帐户 ，因此cicd-bot@my-project.iam.gserviceaccount.com我在角色定义中添加了我认为匹配的权限。

\n

以下是我的最终角色。它具有 configMaps、pods、secrets、services、statefulsets 和thirdPartyObjects 的创建、删除、获取、列出和更新权限，我认为这些权限应该满足要求。

\n

resource "google_project_iam_custom_role" "cicd_bot_role" {\n  project = var.project\n  role_id = "cicd_bot"\n  title   = "CICD Bot"\n  permissions = [\n    "artifactregistry.repositories.downloadArtifacts",\n    "artifactregistry.repositories.uploadArtifacts",\n    "compute.instanceGroupManagers.get",\n    "container.clusters.get",\n    "container.configMaps.create",\n    "container.configMaps.delete",\n    "container.configMaps.get",\n    "container.configMaps.list",\n    "container.configMaps.update",\n    "container.cronJobs.create",\n    "container.cronJobs.delete",\n    "container.cronJobs.get",\n    "container.cronJobs.update",\n    "container.customResourceDefinitions.create",\n    "container.customResourceDefinitions.delete",\n    "container.customResourceDefinitions.get",\n    "container.customResourceDefinitions.list",\n    "container.customResourceDefinitions.update",\n    "container.deployments.create",\n    "container.deployments.delete",\n    "container.deployments.get",\n    "container.deployments.update",\n    "container.ingresses.create",\n    "container.ingresses.delete",\n    "container.ingresses.get",\n    "container.ingresses.update",\n    "container.jobs.create",\n    "container.jobs.delete",\n    "container.jobs.get",\n    "container.jobs.update",\n    "container.namespaces.get",\n    "container.persistentVolumeClaims.create",\n    "container.persistentVolumeClaims.delete",\n    "container.persistentVolumeClaims.get",\n    "container.persistentVolumeClaims.update",\n    "container.pods.create",\n    "container.pods.delete",\n    "container.pods.get",\n    "container.pods.list",\n    "container.pods.update",\n    "container.roleBindings.create",\n    "container.roleBindings.delete",\n    "container.roleBindings.get",\n    "container.roleBindings.update",\n    "container.roles.create",\n    "container.roles.delete",\n    "container.roles.get",\n    "container.roles.update",\n    "container.secrets.create",\n    "container.secrets.delete",\n    "container.secrets.get",\n    "container.secrets.list",\n    "container.secrets.update",\n    "container.serviceAccounts.create",\n    "container.serviceAccounts.delete",\n    "container.serviceAccounts.get",\n    "container.serviceAccounts.update",\n    "container.services.create",\n    "container.services.delete",\n    "container.services.get",\n    "container.services.list",\n    "container.services.update",\n    "container.statefulSets.create",\n    "container.statefulSets.delete",\n    "container.statefulSets.get",\n    "container.statefulSets.list",\n    "container.statefulSets.update",\n    "container.thirdPartyObjects.create",\n    "container.thirdPartyObjects.delete",\n    "container.thirdPartyObjects.get",\n    "container.thirdPartyObjects.list",\n    "container.thirdPartyObjects.update",\n    "dns.changes.create",\n    "dns.changes.get",\n    "dns.resourceRecordSets.get",\n    "dns.resourceRecordSets.list",\n    "dns.resourceRecordSets.update",\n    "storage.buckets.get",\n    "storage.objects.create",\n    "storage.objects.delete",\n    "storage.objects.get",\n    "storage.objects.list",\n  ]\n}\n

\n

然而，部署后，错误仍然存​​在。我想知道是否有必要在 kubernetes 端添加等效权限，因此我也创建了以下 ClusterRole 和 ClusterRoleBinding。

\n

resource "kubernetes_cluster_role" "cicd_bot" {\n  metadata {\n    name = kubernetes_service_account.cicd_bot.metadata[0].name\n  }\n  rule {\n    api_groups = [""]\n    resources  = ["namespaces"]\n    verbs      = ["create", "delete", "get"]\n  }\n  rule {\n    api_groups = [""]\n    resources  = ["configmaps"]\n    verbs      = ["list", "watch", "create", "update", "patch", "get", "delete"]\n  }\n  rule {\n    api_groups = [""]\n    resources  = ["pods"]\n    verbs      = ["list", "watch", "create", "update", "patch", "get", "delete"]\n  }\n  rule {\n    api_groups = [""]\n    resources  = ["secrets"]\n    verbs      = ["list", "watch", "create", "update", "patch", "get", "delete"]\n  }\n  rule {\n    api_groups = [""]\n    resources  = ["services"]\n    verbs      = ["list", "watch", "create", "update", "patch", "get", "delete"]\n  }\n  rule {\n    api_groups = ["apps"]\n    resources  = ["statefulsets"]\n    verbs      = ["list", "watch", "create", "update", "patch", "get", "delete"]\n  }\n  rule {\n    api_groups = ["mongodbcommunity.mongodb.com"]\n    resources  = ["mongodbcommunity"]\n    verbs      = ["list", "watch", "update", "patch", "get"]\n  }\n  rule {\n    api_groups = ["mongodbcommunity.mongodb.com"]\n    resources  = ["mongodbcommunity/finalizers"]\n    verbs      = ["list", "watch", "update", "patch", "get"]\n  }\n  rule {\n    api_groups = ["mongodbcommunity.mongodb.com"]\n    resources  = ["mongodbcommunity/spec"]\n    verbs      = ["list", "watch", "update", "patch", "get"]\n  }\n  rule {\n    api_groups = ["mongodbcommunity.mongodb.com"]\n    resources  = ["mongodbcommunity/status"]\n    verbs      = ["list", "watch", "update", "patch", "get"]\n  }\n}\n\nresource "kubernetes_cluster_role_binding" "cicd_bot" {\n  metadata {\n    name = kubernetes_service_account.cicd_bot.metadata[0].name\n  }\n  subject {\n    kind      = "ServiceAccount"\n    namespace = kubernetes_service_account.cicd_bot.metadata[0].namespace\n    name      = kubernetes_service_account.cicd_bot.metadata[0].name\n  }\n  role_ref {\n    api_group = "rbac.authorization.k8s.io"\n    kind      = "ClusterRole"\n    name      = kubernetes_cluster_role.cicd_bot.metadata[0].name\n  }\n}\n

\n

不幸的是，管道仍然失败并出现相同的错误。我过去曾经能够克服类似的错误，但这次不行。我缺少什么？

\n

更新：我能够通过将角色附加roles/container.admin到我的服务帐户来成功部署。所以现在我需要弄清楚roles/container.admin我的自定义角色没有哪些权限。

\n

Answer 1

遗憾的是，缺少的一项许可是

container.roles.escalate

即使包括所有其他container.*许可也是不够的；container.roles.escalate仍然需要。

这是不幸的，因为它使集群更容易受到权限升级攻击。如果有更安全的方法来实现这一点，我很想听听。我不会将自己的答案标记为“正确”，因为我对此不满意。但是，嘿，至少它正在工作......