带有 GKE 和外部机密的 Google Secret Manager 错误：无法获取 idbindtoken

Question

我目前正在 GKE 标准集群中迁移到具有外部机密操作员的机密管理器。

我收到以下错误，我正在努力了解为什么会发生这种情况：

failed to create GCP secretmanager client: unable to fetch identitybindingtoken: could not get idbindtoken token, status: 404

我希望这里有人可以帮助解决一些问题，或者知道是什么原因造成的，因为我已经把头发拔出来了，花了一天的大部分时间来解决它！

地形：

resource "google_service_account" "external_secret_sa" {
  account_id  = "external-secrets-sa"
  description = "Service Account for external secrets"
  project     = var.project_id
}

# Gives the external secrets Service account access to token creator
resource "google_project_iam_binding" "external_secret_sa_token_creator" {
  project = var.project_id
  role    = "roles/iam.serviceAccountTokenCreator"
  members = [
    "serviceAccount:${google_service_account.external_secret_sa.email}"
  ]
}
# Gives the external secrets Service account access to the secret manager. 
resource "google_project_iam_binding" "external_secret_sa_access" {
  project = var.project_id
  role    = "roles/secretmanager.secretAccessor"
  members = [
    "serviceAccount:${google_service_account.external_secret_sa.email}"
  ]
}

#Allow kubernetes service account to impersonate GCP service account
resource "google_service_account_iam_binding" "external_secret_sa_bind" {
  service_account_id = google_service_account.external_secret_sa.name
  role               = "roles/iam.workloadIdentityUser"

  members = [
    "serviceAccount:${var.project_id}.svc.id.goog[external-secrets/${google_service_account.external_secret_sa.account_id}]"
  ]
  depends_on = [google_service_account.external_secret_sa]
}

外部秘密的 helm 图表配置（它是另一个图表中的依赖项）

external-secrets:
  installCRDs: true
  serviceAccount:
    create: true
    name: external-secrets-sa
    annotations:
      iam.gke.io/gcp-service-account: "[email protected]"

集群秘密存储：

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: gcp-secret-store
spec:
  provider:
    gcpsm:
      projectID: {{ .Values.projectId }}
      auth:
        workloadIdentity:
          # name of the cluster Location, region or zone
          clusterLocation: {{ .Values.region }}
          # name of the GKE cluster
          clusterName: {{ .Values.gkeCluster }}
          # reference the sa from above
          serviceAccountRef:
            name: {{ index .Values "external-secrets" "serviceAccount" "name" }}
            namespace: {{ .Release.Namespace }}

GKE 池 oauth 范围：

node_pools_oauth_scopes = {
    all = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
      "https://www.googleapis.com/auth/devstorage.read_only",
      "https://www.googleapis.com/auth/cloud-platform"

    ]
  }

如果需要更多信息，请告诉我，我很乐意提供我所能解决的问题。我觉得 GKE 和工作负载身份 SA 之间的绑定没有正常工作，但我不确定这是否是 GKE 集群或其他地方的问题。

Answer 1

当我运行定义了不正确的集群名称的秘密存储时，我遇到了同样的问题。 我会更新你的配置，例如：

• 在秘密存储将运行的位置添加 ns（在元数据中）
• 添加将运行秘密存储的 ProjectID（在 wokrloadIdentity 中）

请注意，第一个 ProjectID 引用是 GCPSM 运行的位置，第二个是 Secret Store 运行的位置，这可以不同。

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: gcp-secret-store
  namespace: {{ .Release.Namespace }}
spec:
  provider:
    gcpsm:
      projectID: {{ .Values.projectId }}
      auth:
        workloadIdentity:
          # name of the cluster Location, region or zone
          clusterLocation: {{ .Values.region }}
          # name of the GKE cluster
          clusterName: {{ .Values.gkeCluster }}
          #name of the ProjectID where SecretStore is running
          projectID: {{ .Values.projectId }}
          # reference the sa from above
          serviceAccountRef:
            name: {{ index .Values "external-secrets" "serviceAccount" "name" }}