Configuring GCP Authentication

This guide covers how to securely configure Google Cloud Platform authentication for Controlinfra to scan your infrastructure.

Quick Setup Checklist

For GCE/GKE runners with Workload Identity, you need:

Enable Workload Identity on your GCE instance or GKE cluster
Grant IAM roles to the service account:
- roles/viewer on the project
- roles/storage.objectViewer on the Terraform state bucket
Configure environment variables (optional for Workload Identity):
```
GOOGLE_CLOUD_PROJECT=your-project-id
```
Ensure Application Default Credentials are available on the runner

Overview

Controlinfra needs GCP credentials to:

Access your Terraform state (if using GCS backend)
Run terraform plan to detect drift
Query GCP APIs to compare actual vs. desired state

Required GCP APIs

Before setting up authentication, ensure these APIs are enabled in your GCP project:

API	Purpose	Enable Link
Identity and Access Management (IAM) API	Create service accounts, manage IAM bindings	Enable
Cloud Resource Manager API	Manage project-level IAM policies	Enable
Compute Engine API	For GCE-based runners	Enable
Cloud Storage API	Access Terraform state in GCS	Enable

Enable APIs via CLI:

bash

gcloud services enable \
  iam.googleapis.com \
  cloudresourcemanager.googleapis.com \
  compute.googleapis.com \
  storage.googleapis.com \
  --project=YOUR_PROJECT_ID

API Propagation

After enabling APIs, wait 1-2 minutes for changes to propagate before proceeding.

Authentication Methods

Controlinfra supports two authentication methods for GCP:

Method	Runner Type	Security Level	Best For
Service Account	Cloud or Self-Hosted	Standard	Quick setup, CI/CD pipelines
Workload Identity	Self-Hosted Only	Highest	GCE/GKE runners, production

Option 1: Service Account Key

Use a GCP Service Account with a JSON key file.

When to Use

Getting started quickly
Cloud runner (Controlinfra-managed)
CI/CD pipelines
Development/testing environments

Setup Steps

Step 1: Create a Service Account

bash

# Set your project
export PROJECT_ID=your-project-id

# Create the service account
gcloud iam service-accounts create controlinfra-scanner \
  --display-name="Controlinfra Scanner" \
  --project=$PROJECT_ID

Step 2: Grant Required Roles

bash

# Grant Viewer role for reading resources
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:controlinfra-scanner@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/viewer"

# Grant Storage Object Viewer for Terraform state bucket
gcloud storage buckets add-iam-policy-binding gs://YOUR_STATE_BUCKET \
  --member="serviceAccount:controlinfra-scanner@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/storage.objectViewer"

Step 3: Create and Download Key

bash

# Create JSON key file
gcloud iam service-accounts keys create ~/controlinfra-key.json \
  --iam-account=controlinfra-scanner@${PROJECT_ID}.iam.gserviceaccount.com

Save the Key Securely

The JSON key file contains sensitive credentials. Store it securely and never commit it to source control.

Step 4: Configure in Controlinfra

Option A: Import JSON File (Recommended)

Go to Add Repository → Cloud Provider → GCP
Select "Service Account" authentication
Click "Import from JSON" and select your key file
The Project ID, Client Email, and Private Key will be auto-filled

Option B: Enter Credentials Manually

Go to Add Repository → Cloud Provider → GCP
Select "Service Account" authentication
Enter your credentials:
- Project ID: Your GCP project ID
- Client Email: The service account email (e.g., controlinfra-scanner@project.iam.gserviceaccount.com)
- Private Key: The private key from the JSON file (including -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY-----)

CLI Example

bash

# Using JSON file (recommended)
controlinfra repos add owner/repo \
  --cloud-provider gcp \
  --gcp-auth-method service_account \
  --gcp-json-file /path/to/service-account.json \
  --terraform-dir infrastructure-gcp/

# Using individual credentials
controlinfra repos add owner/repo \
  --cloud-provider gcp \
  --gcp-auth-method service_account \
  --gcp-project-id my-project-id \
  --gcp-client-email terraform@my-project.iam.gserviceaccount.com \
  --gcp-private-key "$(cat key.pem)"

Security Considerations

Credentials are encrypted at rest with AES-256-GCM
Rotate service account keys regularly (recommended: every 90 days)
Use least-privilege permissions
Consider using Workload Identity for production

Option 2: Workload Identity Recommended

Use the identity attached to your GCE instance or GKE pod. No credentials to manage!

Self-Hosted Runner Required

This option is only available when using a self-hosted runner on GCE or GKE.

When to Use

Self-hosted runner on GCE VM or GKE
Enhanced security requirements
No credential rotation needed
Production environments

How It Works

┌─────────────────────┐     ┌──────────────────┐     ┌─────────────────┐
│  Controlinfra       │────▶│  Self-Hosted     │────▶│  GCP APIs       │
│  Cloud              │     │  Runner (GCE)    │     │                 │
└─────────────────────┘     └──────────────────┘     └─────────────────┘
                                    │
                            Attached Service Account
                            (automatic credentials)

Setup Steps for GCE

Step 1: Create Service Account for the VM

bash

export PROJECT_ID=your-project-id

# Create service account for the runner VM
gcloud iam service-accounts create controlinfra-runner \
  --display-name="Controlinfra Runner" \
  --project=$PROJECT_ID

Step 2: Grant Required IAM Roles

bash

SA_EMAIL="controlinfra-runner@${PROJECT_ID}.iam.gserviceaccount.com"

# Grant Viewer role for reading resources
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/viewer"

# Grant Storage Object Viewer for Terraform state bucket
gcloud storage buckets add-iam-policy-binding gs://YOUR_STATE_BUCKET \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/storage.objectViewer"

Step 3: Create GCE Instance with Service Account

bash

# Create VM with the service account attached
gcloud compute instances create controlinfra-runner \
  --zone=us-central1-a \
  --machine-type=e2-medium \
  --image-family=ubuntu-2204-lts \
  --image-project=ubuntu-os-cloud \
  --service-account=${SA_EMAIL} \
  --scopes=cloud-platform \
  --metadata=enable-oslogin=true

Or attach to an existing VM:

bash

gcloud compute instances set-service-account YOUR_VM_NAME \
  --zone=YOUR_ZONE \
  --service-account=${SA_EMAIL} \
  --scopes=cloud-platform

Step 4: Install the Runner

SSH into your VM and install the Controlinfra runner:

bash

# SSH into the VM
gcloud compute ssh controlinfra-runner --zone=us-central1-a

# Install the runner (follow the setup instructions from Controlinfra)
curl -fsSL https://get.controlinfra.com/runner | bash

Step 5: Configure in Controlinfra

Go to Add Repository → Cloud Provider → GCP
Select "Workload Identity" authentication
Enter your Project ID
Select Runner Type: Self-Hosted
Select your runner

CLI Example

bash

controlinfra repos add owner/repo \
  --cloud-provider gcp \
  --gcp-auth-method workload_identity \
  --gcp-project-id my-project-id \
  --runner-type self-hosted \
  --runner-id <runner-id> \
  --terraform-dir infrastructure-gcp/

Setup Steps for GKE (Workload Identity Federation)

For GKE, use Workload Identity Federation to bind Kubernetes service accounts to GCP service accounts.

Step 1: Enable Workload Identity on GKE

bash

# Enable on new cluster
gcloud container clusters create controlinfra-cluster \
  --zone=us-central1-a \
  --workload-pool=${PROJECT_ID}.svc.id.goog

# Or enable on existing cluster
gcloud container clusters update YOUR_CLUSTER \
  --zone=YOUR_ZONE \
  --workload-pool=${PROJECT_ID}.svc.id.goog

Step 2: Create and Configure Service Accounts

bash

# Create GCP service account
gcloud iam service-accounts create controlinfra-runner \
  --display-name="Controlinfra Runner"

# Grant IAM roles (same as GCE)
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:controlinfra-runner@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/viewer"

# Allow Kubernetes SA to impersonate GCP SA
gcloud iam service-accounts add-iam-policy-binding \
  controlinfra-runner@${PROJECT_ID}.iam.gserviceaccount.com \
  --member="serviceAccount:${PROJECT_ID}.svc.id.goog[controlinfra/runner]" \
  --role="roles/iam.workloadIdentityUser"

Step 3: Annotate Kubernetes Service Account

yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: runner
  namespace: controlinfra
  annotations:
    iam.gke.io/gcp-service-account: controlinfra-runner@YOUR_PROJECT.iam.gserviceaccount.com

Benefits

No credentials stored - GCP SDK automatically uses metadata service
No rotation needed - GCP manages credentials automatically
Audit-friendly - All actions tracked under the service account
Secure - Credentials never leave your GCP environment

Required GCP Permissions

Controlinfra needs permissions to read GCP resources and Terraform state.

Recommended: Built-in Roles

Role	Scope	Purpose
roles/viewer	Project	Read-only access to GCP resources
roles/storage.objectViewer	State Bucket	Read Terraform state

bash

# Viewer role on the project
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/viewer"

# Storage Object Viewer on state bucket
gcloud storage buckets add-iam-policy-binding gs://YOUR_BUCKET \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/storage.objectViewer"

Alternative: Custom Role

For more restrictive access, create a custom role:

yaml

title: Controlinfra Drift Scanner
description: Read-only access for drift detection
stage: GA
includedPermissions:
  - compute.instances.get
  - compute.instances.list
  - compute.networks.get
  - compute.networks.list
  - compute.subnetworks.get
  - compute.subnetworks.list
  - compute.firewalls.get
  - compute.firewalls.list
  - storage.buckets.get
  - storage.buckets.list
  - storage.objects.get
  - storage.objects.list
  - pubsub.topics.get
  - pubsub.topics.list
  - pubsub.subscriptions.get
  - pubsub.subscriptions.list
  - cloudsql.instances.get
  - cloudsql.instances.list
  - container.clusters.get
  - container.clusters.list

Create the custom role:

bash

gcloud iam roles create controlinfra_scanner \
  --project=$PROJECT_ID \
  --file=custom-role.yaml

Terraform Backend Configuration

When using GCS as your Terraform backend:

hcl

terraform {
  backend "gcs" {
    bucket = "my-terraform-state"
    prefix = "prod"
  }
}

Ensure the service account has roles/storage.objectViewer on the bucket.

Environment Variables Reference

Service Account Authentication

bash

GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
# Or
GOOGLE_CLOUD_PROJECT=your-project-id

Workload Identity Authentication

No environment variables needed - credentials are automatically provided by the metadata service.

Optionally set:

bash

GOOGLE_CLOUD_PROJECT=your-project-id

Security Best Practices

1. Use Workload Identity for Production

Eliminates credential management
Automatic rotation handled by GCP
Better audit trail

2. Use Least Privilege

Only grant permissions for resources Terraform manages:

bash

# Scope to specific folder or project
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/viewer" \
  --condition='expression=resource.name.startsWith("projects/${PROJECT_ID}/zones/us-central1"),title=us-central1-only'

3. Rotate Service Account Keys

If using Service Account keys, rotate regularly:

bash

# List existing keys
gcloud iam service-accounts keys list \
  --iam-account=${SA_EMAIL}

# Create new key
gcloud iam service-accounts keys create new-key.json \
  --iam-account=${SA_EMAIL}

# Update Controlinfra with new key
# Then delete old key
gcloud iam service-accounts keys delete KEY_ID \
  --iam-account=${SA_EMAIL}

4. Monitor Credential Usage

Enable Cloud Audit Logs:

bash

# View recent activity
gcloud logging read "protoPayload.authenticationInfo.principalEmail=${SA_EMAIL}" \
  --limit=50 \
  --format="table(timestamp, protoPayload.methodName)"

Troubleshooting

"Permission Denied" Error

Error: googleapi: Error 403: Access denied

Verify the service account has required roles:

bash

gcloud projects get-iam-policy $PROJECT_ID \
  --filter="bindings.members:${SA_EMAIL}" \
  --format="table(bindings.role)"

Check if the API is enabled:
bash
```
gcloud services list --enabled
```
Wait a few minutes - IAM changes can take up to 60 seconds to propagate

"Could not find default credentials" Error

Error: google: could not find default credentials

For Service Account:

Ensure GOOGLE_APPLICATION_CREDENTIALS is set
Verify the JSON file exists and is readable

For Workload Identity:

Ensure the VM has a service account attached

Check scopes include cloud-platform:

bash

gcloud compute instances describe YOUR_VM \
  --zone=YOUR_ZONE \
  --format="value(serviceAccounts[].scopes)"

State File Access Issues

Error: Failed to get existing workspaces: storage: object doesn't exist

Verify the bucket and object exist:
bash
```
gcloud storage ls gs://YOUR_BUCKET/
```
Check Storage Object Viewer role on the bucket
Ensure the Terraform backend configuration matches

Metadata Server Not Available

Error: metadata: GCE metadata "instance/service-accounts/default/token" not defined

This means Workload Identity is not properly configured:

Verify the VM has a service account:

bash

gcloud compute instances describe YOUR_VM \
  --format="value(serviceAccounts[].email)"

For GKE, verify Workload Identity is enabled:

bash

kubectl describe serviceaccount runner -n controlinfra

Multi-Project Setup

For scanning infrastructure across multiple GCP projects:

Option 1: Cross-Project IAM

Grant the service account roles in multiple projects:

bash

# Grant in project 1
gcloud projects add-iam-policy-binding project-1 \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/viewer"

# Grant in project 2
gcloud projects add-iam-policy-binding project-2 \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/viewer"

Option 2: Service Account Impersonation

Use one service account to impersonate others:

bash

# Allow impersonation
gcloud iam service-accounts add-iam-policy-binding \
  target-sa@project-2.iam.gserviceaccount.com \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/iam.serviceAccountTokenCreator"

Next Steps

Set Up Self-Hosted Runners - Deploy runners on GCE/GKE
Configure Terraform Backend - Backend settings
Run Your First Scan - Start scanning

Configuring GCP Authentication ​

Overview ​

Required GCP APIs ​

Authentication Methods ​

Option 1: Service Account Key ​

When to Use ​

Setup Steps ​

Step 1: Create a Service Account ​

Step 2: Grant Required Roles ​

Step 3: Create and Download Key ​

Step 4: Configure in Controlinfra ​

CLI Example ​

Security Considerations ​

Option 2: Workload Identity Recommended ​

When to Use ​

How It Works ​

Setup Steps for GCE ​

Step 1: Create Service Account for the VM ​

Step 2: Grant Required IAM Roles ​

Step 3: Create GCE Instance with Service Account ​

Step 4: Install the Runner ​

Step 5: Configure in Controlinfra ​

CLI Example ​

Setup Steps for GKE (Workload Identity Federation) ​

Step 1: Enable Workload Identity on GKE ​

Step 2: Create and Configure Service Accounts ​

Step 3: Annotate Kubernetes Service Account ​

Benefits ​

Required GCP Permissions ​

Recommended: Built-in Roles ​

Alternative: Custom Role ​

Terraform Backend Configuration ​

Environment Variables Reference ​

Service Account Authentication ​

Workload Identity Authentication ​

Security Best Practices ​

1. Use Workload Identity for Production ​

2. Use Least Privilege ​

3. Rotate Service Account Keys ​

4. Monitor Credential Usage ​

Troubleshooting ​

"Permission Denied" Error ​

"Could not find default credentials" Error ​

State File Access Issues ​

Metadata Server Not Available ​

Multi-Project Setup ​

Option 1: Cross-Project IAM ​

Option 2: Service Account Impersonation ​

Next Steps ​

Configuring GCP Authentication

Overview

Required GCP APIs

Authentication Methods

Option 1: Service Account Key

When to Use

Setup Steps

Step 1: Create a Service Account

Step 2: Grant Required Roles

Step 3: Create and Download Key

Step 4: Configure in Controlinfra

CLI Example

Security Considerations

Option 2: Workload Identity Recommended

When to Use

How It Works

Setup Steps for GCE

Step 1: Create Service Account for the VM

Step 2: Grant Required IAM Roles

Step 3: Create GCE Instance with Service Account

Step 4: Install the Runner

Step 5: Configure in Controlinfra

CLI Example

Setup Steps for GKE (Workload Identity Federation)

Step 1: Enable Workload Identity on GKE

Step 2: Create and Configure Service Accounts

Step 3: Annotate Kubernetes Service Account

Benefits

Required GCP Permissions

Recommended: Built-in Roles

Alternative: Custom Role

Terraform Backend Configuration

Environment Variables Reference

Service Account Authentication

Workload Identity Authentication

Security Best Practices

1. Use Workload Identity for Production

2. Use Least Privilege

3. Rotate Service Account Keys

4. Monitor Credential Usage

Troubleshooting

"Permission Denied" Error

"Could not find default credentials" Error

State File Access Issues

Metadata Server Not Available

Multi-Project Setup

Option 1: Cross-Project IAM

Option 2: Service Account Impersonation

Next Steps