Appearance

Configuring GCP Authentication ​

This guide covers how to securely configure Google Cloud Platform authentication for Controlinfra to scan your infrastructure.

Quick Setup Checklist

For GCE/GKE runners with Workload Identity, you need:

  1. Enable Workload Identity on your GCE instance or GKE cluster
  2. Grant IAM roles to the service account:
    • roles/viewer on the project
    • roles/storage.objectViewer on the Terraform state bucket
  3. Configure environment variables (optional for Workload Identity):
    GOOGLE_CLOUD_PROJECT=your-project-id
  4. Ensure Application Default Credentials are available on the runner

Overview ​

Controlinfra needs GCP credentials to:

  1. Access your Terraform state (if using GCS backend)
  2. Run terraform plan to detect drift
  3. Query GCP APIs to compare actual vs. desired state

Required GCP APIs ​

Before setting up authentication, ensure these APIs are enabled in your GCP project:

APIPurposeEnable Link
Identity and Access Management (IAM) APICreate service accounts, manage IAM bindingsEnable
Cloud Resource Manager APIManage project-level IAM policiesEnable
Compute Engine APIFor GCE-based runnersEnable
Cloud Storage APIAccess Terraform state in GCSEnable

Enable APIs via CLI:

bash
gcloud services enable \
  iam.googleapis.com \
  cloudresourcemanager.googleapis.com \
  compute.googleapis.com \
  storage.googleapis.com \
  --project=YOUR_PROJECT_ID

API Propagation

After enabling APIs, wait 1-2 minutes for changes to propagate before proceeding.

Authentication Methods ​

Controlinfra supports two authentication methods for GCP:

MethodRunner TypeSecurity LevelBest For
Service Account KeyCloud or Self-HostedStandardQuick setup, CI/CD pipelines
Workload IdentitySelf-Hosted OnlyHighestGCE/GKE runners, production

Trust requirements at a glance ​

Each method involves a different principal on Controlinfra's side. Pick the one whose hosting model matches what you can run.

MethodWho Controlinfra authenticates asWhat you trust on Controlinfra's sideWhere the scanner must run
Service Account KeyThe Google Cloud Service Account you created — Controlinfra signs JWTs with the SA's downloaded private key and exchanges them with Google for access tokens.Nothing on Controlinfra's side. The Service Account belongs to your project; Controlinfra just holds its private key in encrypted storage.Cloud runner or self-hosted. Works from Controlinfra-managed infrastructure or your own.
Workload IdentityThe GCP Service Account attached to the runner's compute (GCE VM service account or GKE Workload Identity binding). Tokens come from the GCE metadata server; the private key never leaves Google.Nothing — there is no Controlinfra-issued principal involved.Self-hosted runner only, and it must be running on GCE or GKE with an attached service account. Controlinfra's cloud runner is not hosted on Google Cloud, so picking this with a cloud runner will fail at scan time. The Add Cloud Account form surfaces a warning when you pick Workload Identity without a self-hosted GCP runner.

Why this matters

Workload Identity does not degrade gracefully — it fails outright if Controlinfra's cloud runner is the executor, because the cloud runner has no Google Cloud-side identity to assume. Pick Service Account Key for cloud-runner setups; pick Workload Identity when you operate the runner yourself on GCE/GKE.

Option 1: Service Account Key ​

Use a GCP Service Account with a JSON key file.

Trust requirements

  • Who Controlinfra authenticates as: the Service Account you create below. Controlinfra signs JWTs with the SA's private key and exchanges them with Google's OAuth2 token endpoint.
  • What you trust on Controlinfra's side: nothing — the SA belongs to your GCP project. Controlinfra is just an opaque consumer of its private key.
  • Where this works: cloud runner or self-hosted runner — there is no infrastructure dependency on Controlinfra's side.

When to Use ​

  • Getting started quickly
  • Cloud runner (Controlinfra-managed)
  • CI/CD pipelines
  • Development/testing environments

Setup Steps ​

Step 1: Create a Service Account ​

bash
# Set your project
export PROJECT_ID=your-project-id

# Create the service account
gcloud iam service-accounts create controlinfra-scanner \
  --display-name="Controlinfra Scanner" \
  --project=$PROJECT_ID

Step 2: Grant Required Roles ​

bash
# Grant Viewer role for reading resources
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:controlinfra-scanner@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/viewer"

# Grant Storage Object Viewer for Terraform state bucket
gcloud storage buckets add-iam-policy-binding gs://YOUR_STATE_BUCKET \
  --member="serviceAccount:controlinfra-scanner@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/storage.objectViewer"

Step 3: Create and Download Key ​

bash
# Create JSON key file
gcloud iam service-accounts keys create ~/controlinfra-key.json \
  --iam-account=controlinfra-scanner@${PROJECT_ID}.iam.gserviceaccount.com

Save the Key Securely

The JSON key file contains sensitive credentials. Store it securely and never commit it to source control.

Step 4: Configure in Controlinfra ​

Option A: Import JSON File (Recommended)

  1. Go to Add Repository → Cloud Provider → GCP
  2. Select "Service Account" authentication
  3. Click "Import from JSON" and select your key file
  4. The Project ID, Client Email, and Private Key will be auto-filled

Option B: Enter Credentials Manually

  1. Go to Add Repository → Cloud Provider → GCP
  2. Select "Service Account" authentication
  3. Enter your credentials:
    • Project ID: Your GCP project ID
    • Client Email: The service account email (e.g., controlinfra-scanner@project.iam.gserviceaccount.com)
    • Private Key: The private key from the JSON file (including -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY-----)

CLI Example ​

bash
# Using JSON file (recommended)
controlinfra repos add owner/repo \
  --cloud-provider gcp \
  --gcp-auth-method service_account \
  --gcp-json-file /path/to/service-account.json \
  --terraform-dir infrastructure-gcp/

# Using individual credentials
controlinfra repos add owner/repo \
  --cloud-provider gcp \
  --gcp-auth-method service_account \
  --gcp-project-id my-project-id \
  --gcp-client-email terraform@my-project.iam.gserviceaccount.com \
  --gcp-private-key "$(cat key.pem)"

Security Considerations ​

  • Credentials are encrypted at rest with AES-256-GCM
  • Rotate service account keys regularly (recommended: every 90 days)
  • Use least-privilege permissions
  • Consider using Workload Identity for production

Option 2: Workload Identity Recommended ​

Use the identity attached to your GCE instance or GKE pod. No credentials to manage!

Trust requirements & hosting constraint

  • Who Controlinfra authenticates as: the GCP Service Account attached to the runner's compute. Tokens come from the GCE metadata server (http://metadata.google.internal) and never leave Google's environment.
  • What you trust on Controlinfra's side: nothing — there is no Controlinfra-issued principal involved.
  • Where this works: only on a self-hosted runner running on GCE or GKE with an attached Service Account. Controlinfra's cloud runner is not hosted on Google Cloud, so picking this option with a cloud runner will fail at scan time. The Add Cloud Account form surfaces a warning when you pick Workload Identity without selecting a self-hosted GCP runner.

Self-Hosted Runner Required

This option is only available when using a self-hosted runner on GCE or GKE.

When to Use ​

  • Self-hosted runner on GCE VM or GKE
  • Enhanced security requirements
  • No credential rotation needed
  • Production environments

How It Works ​

┌─────────────────────┐     ┌──────────────────┐     ┌─────────────────┐
│  Controlinfra       │────▶│  Self-Hosted     │────▶│  GCP APIs       │
│  Cloud              │     │  Runner (GCE)    │     │                 │
└─────────────────────┘     └──────────────────┘     └─────────────────┘
                                    │
                            Attached Service Account
                            (automatic credentials)

Setup Steps for GCE ​

Step 1: Create Service Account for the VM ​

bash
export PROJECT_ID=your-project-id

# Create service account for the runner VM
gcloud iam service-accounts create controlinfra-runner \
  --display-name="Controlinfra Runner" \
  --project=$PROJECT_ID

Step 2: Grant Required IAM Roles ​

bash
SA_EMAIL="controlinfra-runner@${PROJECT_ID}.iam.gserviceaccount.com"

# Grant Viewer role for reading resources
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/viewer"

# Grant Storage Object Viewer for Terraform state bucket
gcloud storage buckets add-iam-policy-binding gs://YOUR_STATE_BUCKET \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/storage.objectViewer"

Step 3: Create GCE Instance with Service Account ​

bash
# Create VM with the service account attached
gcloud compute instances create controlinfra-runner \
  --zone=us-central1-a \
  --machine-type=e2-medium \
  --image-family=ubuntu-2204-lts \
  --image-project=ubuntu-os-cloud \
  --service-account=${SA_EMAIL} \
  --scopes=cloud-platform \
  --metadata=enable-oslogin=true

Or attach to an existing VM:

bash
gcloud compute instances set-service-account YOUR_VM_NAME \
  --zone=YOUR_ZONE \
  --service-account=${SA_EMAIL} \
  --scopes=cloud-platform

Step 4: Install the Runner ​

SSH into your VM and install the Controlinfra runner:

bash
# SSH into the VM
gcloud compute ssh controlinfra-runner --zone=us-central1-a

# Install the runner (follow the setup instructions from Controlinfra)
curl -fsSL https://get.controlinfra.com/runner | bash

Step 5: Configure in Controlinfra ​

  1. Go to Add Repository → Cloud Provider → GCP
  2. Select "Workload Identity" authentication
  3. Enter your Project ID
  4. Select Runner Type: Self-Hosted
  5. Select your runner

CLI Example ​

bash
controlinfra repos add owner/repo \
  --cloud-provider gcp \
  --gcp-auth-method workload_identity \
  --gcp-project-id my-project-id \
  --runner-type self-hosted \
  --runner-id <runner-id> \
  --terraform-dir infrastructure-gcp/

Setup Steps for GKE (Workload Identity Federation) ​

For GKE, use Workload Identity Federation to bind Kubernetes service accounts to GCP service accounts.

Step 1: Enable Workload Identity on GKE ​

bash
# Enable on new cluster
gcloud container clusters create controlinfra-cluster \
  --zone=us-central1-a \
  --workload-pool=${PROJECT_ID}.svc.id.goog

# Or enable on existing cluster
gcloud container clusters update YOUR_CLUSTER \
  --zone=YOUR_ZONE \
  --workload-pool=${PROJECT_ID}.svc.id.goog

Step 2: Create and Configure Service Accounts ​

bash
# Create GCP service account
gcloud iam service-accounts create controlinfra-runner \
  --display-name="Controlinfra Runner"

# Grant IAM roles (same as GCE)
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:controlinfra-runner@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/viewer"

# Allow Kubernetes SA to impersonate GCP SA
gcloud iam service-accounts add-iam-policy-binding \
  controlinfra-runner@${PROJECT_ID}.iam.gserviceaccount.com \
  --member="serviceAccount:${PROJECT_ID}.svc.id.goog[controlinfra/runner]" \
  --role="roles/iam.workloadIdentityUser"

Step 3: Annotate Kubernetes Service Account ​

yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: runner
  namespace: controlinfra
  annotations:
    iam.gke.io/gcp-service-account: controlinfra-runner@YOUR_PROJECT.iam.gserviceaccount.com

Benefits ​

  • No credentials stored - GCP SDK automatically uses metadata service
  • No rotation needed - GCP manages credentials automatically
  • Audit-friendly - All actions tracked under the service account
  • Secure - Credentials never leave your GCP environment

Option 3: Workload Identity Federation (Controlinfra-hosted) ​

This is the recommended path when Controlinfra runs the scan against your GCP project from our control plane (no self-hosted runner). It works the same way AWS and Azure OIDC do for our SaaS: Controlinfra signs a short-lived JWT, GCP STS exchanges it for a federated access token, and that token impersonates a service account in your project. No long-lived credentials are stored on our side.

When to Use ​

  • You don't have / don't want a self-hosted runner in GCP
  • You want to mirror the AWS/Azure OIDC setup for consistency
  • Your security policy forbids storing service account JSON keys

How It Works ​

  1. You create a Workload Identity Pool + OIDC Provider in your GCP project, trusting OIDC_ISSUER (Controlinfra's issuer URL).
  2. You create a Service Account in the same project with the required scan/discovery permissions.
  3. You grant roles/iam.workloadIdentityUser on that SA so federated identities matching sub == "org:<your-org-id>" can impersonate it.
  4. You paste the audience (pool/provider resource name) and SA email into the Controlinfra GCP credentials form. We use validateGcpCredentials to test the full chain immediately.

Setup Steps ​

Replace PROJECT_ID, PROJECT_NUMBER, POOL, PROVIDER, ORG_ID with your values. Controlinfra's issuer is the value of OIDC_ISSUER (visible to your account team — typically https://api.controlinfra.com or your dedicated deployment hostname).

bash
# 1. Create the workload identity pool + OIDC provider.
gcloud iam workload-identity-pools create POOL \
  --project=PROJECT_ID --location=global \
  --display-name="Controlinfra"

gcloud iam workload-identity-pools providers create-oidc PROVIDER \
  --project=PROJECT_ID --location=global \
  --workload-identity-pool=POOL \
  --issuer-uri="https://api.controlinfra.com" \
  --attribute-mapping="google.subject=assertion.sub" \
  --attribute-condition="assertion.sub.startsWith('org:ORG_ID')"

# 2. Create the SA Controlinfra will impersonate.
gcloud iam service-accounts create controlinfra-scanner \
  --project=PROJECT_ID --display-name="Controlinfra scanner"

# 3. Allow our federated identity to impersonate it.
gcloud iam service-accounts add-iam-policy-binding \
  controlinfra-scanner@PROJECT_ID.iam.gserviceaccount.com \
  --project=PROJECT_ID --role="roles/iam.workloadIdentityUser" \
  --member="principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL/subject/org:ORG_ID"

# 4. Grant the SA scan / discovery roles (Required Permissions section below).
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:controlinfra-scanner@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/viewer"
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:controlinfra-scanner@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/cloudasset.viewer"

What to Paste in the Form ​

FieldValue
Project IDPROJECT_ID
Audience//iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL/providers/PROVIDER
Service Account Emailcontrolinfra-scanner@PROJECT_ID.iam.gserviceaccount.com

The form's audience field expects the full //iam.googleapis.com/... resource name (note the leading double slash) — gcloud iam workload-identity-pools providers describe shows it under name.

Validation ​

Saving the form runs validateGcpCredentials, which:

  1. Mints a JWT with the correct audience and signs it with our OIDC key.
  2. Calls GCP STS https://sts.googleapis.com/v1/token.
  3. Impersonates the SA via IAM Credentials generateAccessToken.
  4. Calls cloudresourcemanager.projects.get to confirm the SA can see the project.

If any step fails, the form surfaces the specific error so you know whether to fix the audience, the SA binding, or the project-level role grants.

Troubleshooting ​

  • "STS token-exchange failed (400): invalid argument" — the audience doesn't match the pool/provider configured in your project, or the attribute-condition rejected our sub claim. Verify the audience string exactly and that the condition matches org:<your-org-id>.
  • "SA impersonation failed (403)" — the roles/iam.workloadIdentityUser binding is missing or names a different principal. The member=principal://... line is the most error-prone — double-check it against your real PROJECT_NUMBER and ORG_ID.
  • "Permission denied on projects.get" — federation worked but the SA lacks roles/viewer (or equivalent) on the project. Run step 4 again.

Required GCP Permissions ​

Controlinfra needs permissions to read GCP resources and Terraform state.

RoleScopePurpose
roles/viewerProjectRead-only access to GCP resources
roles/storage.objectViewerState BucketRead Terraform state
bash
# Viewer role on the project
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/viewer"

# Storage Object Viewer on state bucket
gcloud storage buckets add-iam-policy-binding gs://YOUR_BUCKET \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/storage.objectViewer"

Alternative: Custom Role ​

For more restrictive access, create a custom role:

yaml
title: Controlinfra Drift Scanner
description: Read-only access for drift detection
stage: GA
includedPermissions:
  - compute.instances.get
  - compute.instances.list
  - compute.networks.get
  - compute.networks.list
  - compute.subnetworks.get
  - compute.subnetworks.list
  - compute.firewalls.get
  - compute.firewalls.list
  - storage.buckets.get
  - storage.buckets.list
  - storage.objects.get
  - storage.objects.list
  - pubsub.topics.get
  - pubsub.topics.list
  - pubsub.subscriptions.get
  - pubsub.subscriptions.list
  - cloudsql.instances.get
  - cloudsql.instances.list
  - container.clusters.get
  - container.clusters.list

Create the custom role:

bash
gcloud iam roles create controlinfra_scanner \
  --project=$PROJECT_ID \
  --file=custom-role.yaml

Terraform Backend Configuration ​

When using GCS as your Terraform backend:

hcl
terraform {
  backend "gcs" {
    bucket = "my-terraform-state"
    prefix = "prod"
  }
}

Ensure the service account has roles/storage.objectViewer on the bucket.

Environment Variables Reference ​

Service Account Authentication ​

bash
GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
# Or
GOOGLE_CLOUD_PROJECT=your-project-id

Workload Identity Authentication ​

No environment variables needed - credentials are automatically provided by the metadata service.

Optionally set:

bash
GOOGLE_CLOUD_PROJECT=your-project-id

Security Best Practices ​

1. Use Workload Identity for Production ​

  • Eliminates credential management
  • Automatic rotation handled by GCP
  • Better audit trail

2. Use Least Privilege ​

Only grant permissions for resources Terraform manages:

bash
# Scope to specific folder or project
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/viewer" \
  --condition='expression=resource.name.startsWith("projects/${PROJECT_ID}/zones/us-central1"),title=us-central1-only'

3. Rotate Service Account Keys ​

If using Service Account keys, rotate regularly:

bash
# List existing keys
gcloud iam service-accounts keys list \
  --iam-account=${SA_EMAIL}

# Create new key
gcloud iam service-accounts keys create new-key.json \
  --iam-account=${SA_EMAIL}

# Update Controlinfra with new key
# Then delete old key
gcloud iam service-accounts keys delete KEY_ID \
  --iam-account=${SA_EMAIL}

4. Monitor Credential Usage ​

Enable Cloud Audit Logs:

bash
# View recent activity
gcloud logging read "protoPayload.authenticationInfo.principalEmail=${SA_EMAIL}" \
  --limit=50 \
  --format="table(timestamp, protoPayload.methodName)"

Troubleshooting ​

"Permission Denied" Error ​

Error: googleapi: Error 403: Access denied

  1. Verify the service account has required roles:

    bash
    gcloud projects get-iam-policy $PROJECT_ID \
  --filter="bindings.members:${SA_EMAIL}" \
  --format="table(bindings.role)"

  2. Check if the API is enabled:

    bash
    gcloud services list --enabled

  3. Wait a few minutes - IAM changes can take up to 60 seconds to propagate

"Could not find default credentials" Error ​

Error: google: could not find default credentials

For Service Account:

  • Ensure GOOGLE_APPLICATION_CREDENTIALS is set
  • Verify the JSON file exists and is readable

For Workload Identity:

  • Ensure the VM has a service account attached
  • Check scopes include cloud-platform:
    bash
    gcloud compute instances describe YOUR_VM \
  --zone=YOUR_ZONE \
  --format="value(serviceAccounts[].scopes)"

State File Access Issues ​

Error: Failed to get existing workspaces: storage: object doesn't exist

  1. Verify the bucket and object exist:

    bash
    gcloud storage ls gs://YOUR_BUCKET/

  2. Check Storage Object Viewer role on the bucket

  3. Ensure the Terraform backend configuration matches

Metadata Server Not Available ​

Error: metadata: GCE metadata "instance/service-accounts/default/token" not defined

This means Workload Identity is not properly configured:

  1. Verify the VM has a service account:

    bash
    gcloud compute instances describe YOUR_VM \
  --format="value(serviceAccounts[].email)"

  2. For GKE, verify Workload Identity is enabled:

    bash
    kubectl describe serviceaccount runner -n controlinfra

Multi-Project Setup ​

For scanning infrastructure across multiple GCP projects:

Option 1: Cross-Project IAM ​

Grant the service account roles in multiple projects:

bash
# Grant in project 1
gcloud projects add-iam-policy-binding project-1 \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/viewer"

# Grant in project 2
gcloud projects add-iam-policy-binding project-2 \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/viewer"

Option 2: Service Account Impersonation ​

Use one service account to impersonate others:

bash
# Allow impersonation
gcloud iam service-accounts add-iam-policy-binding \
  target-sa@project-2.iam.gserviceaccount.com \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/iam.serviceAccountTokenCreator"

Next Steps ​