gcp-emulator-provisioner
Pub/Sub and Cloud Storage state for local GCP emulators — and the same config runs against real Google Cloud.
- v2.0.1 · latest
- MIT
- Go
- same config runs against real Google Cloud
docker pull ghcr.io/datarocks-ag/gcp-emulator-provisioner:2.0.1What it does
- Topics, subscriptions, buckets — Idempotent provisioning of Pub/Sub topics and subscriptions and Cloud Storage buckets.
- Drift reconciliation — Sends a minimal update mask: only the fields that actually changed.
- Fixed-at-creation fields — Fields Pub/Sub fixes at creation (filter, enable_message_ordering) are reported as warnings instead of failing the run.
- Emulator gaps — Gaps in the emulators that Google Cloud does not have are downgraded to warnings, so one config works against both.
- Dry run — Previews every mutation against live state.
- Never destructive — Never deletes topics, subscriptions or buckets that are not in the config.
project_id: local-dev
# Service account key files for local development.
#
# The key is generated here and authenticates to nothing — Google holds no
# matching public key. It exists because many libraries (Spring Cloud GCP among
# them) parse GOOGLE_APPLICATION_CREDENTIALS eagerly at startup and refuse to
# run without a structurally valid key, even when every call they make is
# answered by an emulator.
#
# The file is never rewritten once it exists: its contents are a fresh keypair,
# so rewriting would rotate the credential underneath whatever already loaded
# it. Set `strategy: update` on the entry to force a new one.
credentials:
- path: /secrets/fake-sa.json
# Service account id in the generated address. Default: local-emulator.
account: local-emulator
# Points every OAuth URL in the file at one address, so a library that does
# try to mint a token reaches the local stub instead of accounts.google.com.
# This is what a JVM application needs: google-cloud-storage for Java has no
# STORAGE_EMULATOR_HOST equivalent, so it authenticates for real and the
# exchange has to land somewhere local. Omit to use Google's endpoints.
token_uri: http://gcp-token-stub:8099/token
pubsub:
topics:
- name: orders
labels:
domain: commerce
# How long the topic itself retains messages, so a new subscription can
# seek back into the past. Omit to leave retention to the subscriptions.
message_retention: 24h
subscriptions:
- name: order-processor
# 10s-600s, and a whole number of seconds: Pub/Sub stores this as
# ack_deadline_seconds, so a fractional value would be truncated and
# then reported as drift on every subsequent run.
ack_deadline: 30s
# 10m-31d.Full documentation, configuration reference and how-tos.
Read the docs →More provisioners
keycloak-provisionerrealms, clients, protocol mappers, roles, userspostgres-provisionerroles, databases, schemas, privileges, extensionsnats-provisionerstreams, consumersredpanda-provisionertopics, schemas, SASL users, ACLsseaweedfs-provisionerS3 buckets, IAM identities, bucket policies, lifecycle rules
Data Rocks AG
[ONE OR TWO LINES ON WHAT DATA ROCKS DOES AND FOR WHOM]