Secret Manager
Secret Manager is managed storage for the small chunks of sensitive data that applications need at runtime — API keys, database passwords, TLS certificates, JWT signing keys, third-party credentials. Each secret has versions, IAM-controlled access, and an auditable access trail. The pattern it replaces is the one every engineer has seen before — secrets in environment variables, secrets in config files, secrets in source code, secrets pasted into wikis.
The single most common confusion in this chapter is between Secret Manager and Cloud KMS, the next service. They sound similar but solve different problems. Secret Manager holds the secret value itself. KMS holds cryptographic keys that encrypt and decrypt other data. The two are complementary; using one when the other is the right tool is a frequent architectural error.
The Secret Storage Model
A secret is a named container; a version is an immutable payload at a point in time. The current version holds the active value; older versions stay around for audit and rollback unless explicitly destroyed. Applications fetch by secret name plus version (or the latest alias) and receive the bytes. The secret name lives in your configuration and source code; the secret value never does.
Payloads are arbitrary bytes — Secret Manager does not interpret content. Most secrets are short ASCII strings (API keys, passwords) but anything fits as long as it is under the size limit (currently 64 KiB).
Versions and Aliases
Every time a secret is updated, Secret Manager appends a new version with a monotonically increasing number. The application can pin to a specific version (secret-name/versions/5) or use the latest alias to always get the most recent. Both have a place. Pinned versions are deterministic and safer in production — a deploy is the only thing that changes which version runs. The latest alias is convenient but means a secret rotation can change application behavior between requests; that is the point sometimes, but it should be a conscious choice.
IAM Access Control and Audit
Each secret is its own resource with its own IAM policy. The most common bindings: roles/secretmanager.secretAccessor on the service account that needs to read the secret, and roles/secretmanager.admin on the team that manages it. Bind to groups, not individuals — the same rule as everywhere else in IAM.
Admin Activity audit logs — writes and policy changes — are always on for Secret Manager. The read trail is different: each AccessSecretVersion call is a Data Access log, which is off by default and must be enabled explicitly. Enable it — when an incident happens, answering "who accessed this credential and when" with a query is one of the strongest reasons to use Secret Manager over ad-hoc storage.
Rotation
Long-lived secrets rot. The credential for a third-party API today is the credential that was checked into a wiki six months ago. Secret Manager supports rotation through a Pub/Sub message published on a configurable schedule; a Cloud Function or Cloud Run service subscribes, generates a new credential (mints an API key via the partner's API, rotates a database password, etc.), and writes it as a new version. The application picks up the new version automatically if it uses the latest alias, or after the next deploy if it pins.
Replication
Two modes. Automatic replication stores the secret in multiple regions automatically — the right default, no thought required. User-managed replication specifies the exact regions; useful for data residency requirements where a secret must remain inside a specific country or jurisdiction. Pick user-managed only when residency is a real constraint; automatic is simpler and more resilient.
Secret Manager vs Cloud KMS
The two services sound similar enough that they are routinely confused. The mental model: Secret Manager stores secrets you read at runtime; KMS stores keys you use to encrypt other data.
Secret Manager — the secret value lives here. API keys, passwords, certificates, JWT signing keys. The application reads the value, uses it, and discards it. The secret is the data.
Cloud KMS — cryptographic keys live here, not data. Your application uses a KMS key to encrypt or decrypt other data (envelope encryption); the key itself never leaves KMS. The key is a tool, not the payload.
- Secrets stored in environment variables, config files, or source code. Every deploy is a leak risk; every contributor sees them.
- Secret Manager used as envelope encryption — encrypting your own data with a key you also keep there. That is KMS's job.
- Cloud KMS used to "store secrets" by wrapping them. KMS holds keys; Secret Manager holds the secret value. Wrapping a secret with a KMS key just adds steps without changing the answer.
- No rotation strategy for long-lived secrets. The 2-year-old database password sits in production indefinitely, with every former employee who ever read it still able to use it.
- IAM bindings on each individual secret instead of groups. Twenty engineers join, twenty secrets each get a binding update; one engineer leaves, twenty bindings need cleanup.
- The
latestalias used in production code where deterministic behavior matters. A rotation between requests can change application behavior without a deploy event.
- Secret Manager for all credentials. Environment variables, config files, and source code never hold real secrets in production.
- IAM bindings to groups, not to individual service accounts where multiple workloads share the same secret access pattern.
- Pinned versions in production code paths;
latestonly where the application explicitly wants live rotation behavior. - Rotation callbacks for any third-party credential that can be programmatically rotated. Database passwords, API keys, certificates.
- Data Access (DATA_READ) audit logging enabled up front and reviewed during any security incident — the access trail is the single biggest reason to use Secret Manager in the first place.
- Automatic replication unless data residency requires user-managed. Most secrets do not have residency constraints; do not invent ones that aren't there.
- Client-side caching of fetched secrets with a sensible TTL. Calling Secret Manager on every request wastes calls and adds latency.
Knowledge Check
What is the principal difference between Secret Manager and Cloud KMS?
- They are two names for the same underlying service; KMS is simply the older brand name that is being phased out
- Secret Manager stores the secret value (API keys, passwords); Cloud KMS stores cryptographic keys you use to encrypt other data — the key itself never leaves KMS
- Secret Manager handles only symmetric secrets, while Cloud KMS is the service that handles asymmetric secrets
- Secret Manager is strictly a single-region service only, whereas Cloud KMS is the one that supports fully global cross-region replication of all of its stored data
When should production code pin to a specific secret version instead of using the latest alias?
- Always, in every case — the
latestalias is fully deprecated and is only kept around now for backward compatibility - When deterministic behavior matters and a rotation should not change application behavior without a deploy event — pinning makes the deploy the only thing that changes which version is live
- When the secret holds a binary TLS certificate rather than a string, since the
latestalias only ever resolves correctly for plain textual secret values and silently fails on binary payloads - Pinning is required for any secret larger than 4 KB, because the
latestalias is limited to small payloads only
How does Secret Manager support automatic credential rotation?
- Secret Manager itself generates a brand-new value on a fixed schedule and then atomically swaps it in server-side
- A rotation schedule publishes a Pub/Sub message; a Cloud Function or Cloud Run service generates a new credential and writes it as a new version
- Rotation is not supported in Secret Manager at all; teams have to manually delete and then recreate the secret each time
- A built-in cron job inside Secret Manager periodically updates the value on its own using configured rotation rules that are defined per secret type
A team stores PostgreSQL passwords directly in a Cloud KMS key payload. What is wrong with this pattern?
- Cloud KMS does not accept raw passwords as payloads at all; it is only able to store symmetric encryption keys
- KMS holds keys, not secret values — the actual secret should live in Secret Manager, where IAM, audit, and rotation work the way they were designed for credentials
- Cloud KMS bills per stored byte at a substantially higher rate than Secret Manager does for the same data
- KMS key payloads cannot exceed a hard 4 KB size cap, which is far too small to hold most of the real database connection passwords that teams actually use day to day
Why is the Cloud Audit Logs access trail such an important part of Secret Manager?
- Audit Logs must be enabled first before you are allowed to set any IAM bindings on secrets
- When a security incident occurs, the access trail tells you exactly who read the credential and when — a question that is unanswerable for secrets stored in environment variables, config files, or wikis
- Audit Logs automatically auto-rotate any secret that has been accessed more than 100 separate times within a window, generating a fresh new version once that access-count threshold is crossed by any caller
- Audit Logs are what unlock the multi-region automatic replication mode for a secret
You got correct