GCP Source Plugin

The GCP Source plugin for CloudQuery extracts configuration from a variety of GCP APIs and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Libraries in Use

• https://cloud.google.com/go/docs/reference (opens in a new tab)
• https://github.com/googleapis/google-api-go-client (opens in a new tab)

Authentication

The GCP plugin authenticates using your Application Default Credentials (opens in a new tab). Available options are all the same options described here (opens in a new tab) in detail:

Local Environment:

• gcloud auth application-default login (recommended when running locally)

Google Cloud cloud-based development environment:

• When you run on Cloud Shell or Cloud Code credentials are already available.

Google Cloud containerized environment:

• When running on GKE use workload identity (opens in a new tab).

Google Cloud services that support attaching a service account (opens in a new tab):

• Services such as Compute Engine, App Engine and functions supporting attaching a user-managed service account which will CloudQuery will be able to utilize.

On-premises or another cloud provider

• The suggested way is to use Workload identity federation (opens in a new tab)
• If not available you can always use service account keys and export the location of the key via GOOGLE_APPLICATION_CREDENTIALS. Highly not recommended as long-lived keys are a security risk

Query Examples:

Find all buckets without uniform bucket-level access

select project_id, name from gcp_storage_buckets where uniform_bucket_level_access->>'Enabled' = 'true';