Secrets Management
Kubernetes Secrets are not secure by default — they are base64-encoded and, without extra configuration, sit in etcd in plaintext. Making them genuinely safe takes three things: encryption at rest, tight RBAC, and often an external secret store that keeps the sensitive value out of etcd entirely.
This topic is the security-chapter follow-through on ConfigMaps and Secrets (Topic 10). The mechanics of using a Secret are simple; protecting it is the part that actually matters.
Base64 Is Not Encryption
A Secret's values are base64-encoded, which is reversible by anyone — encoding, not encryption. By default the API server writes Secrets to etcd as-is, so anyone who can read etcd (a node backup, a compromised control-plane host) can read every secret. And anyone with get secrets RBAC in a namespace can read them through the API. The convenience of Secrets is real; the security is not automatic.
Encryption at Rest
The first fix is encryption at rest: configure the API server with an EncryptionConfiguration so Secrets are encrypted before being written to etcd. The strong form uses a KMS provider — a cloud or external key-management service — so the data-encryption keys are themselves protected by a key the cluster never stores in plaintext. With KMS encryption, an etcd dump no longer hands over your secrets. On managed clusters this is often available as a checkbox tied to the cloud KMS.
RBAC on Secrets
Encryption protects data at rest; RBAC protects it in use. Read access to Secrets should be tightly scoped — few identities, few namespaces. The get, list, and watch verbs on secrets are high-value grants; audit who has them. Each workload's ServiceAccount should be able to read only the Secrets it needs, not every Secret in its namespace.
External Secret Stores
The strongest posture keeps the sensitive value out of etcd altogether. An external secret store — HashiCorp Vault, or a cloud secrets manager — holds the real secret, and Kubernetes integrates via the External Secrets Operator (syncs into a Secret on demand) or the Secrets Store CSI driver (mounts directly into the Pod). This adds rotation, audit, and central control, and means a cluster compromise does not expose the source of truth. For high-value credentials, this is the recommended pattern.
| Approach | Where the secret lives | Strength |
|---|---|---|
| Plain Secret | etcd, base64 only | Weak — avoid for real secrets |
| Encryption at rest (KMS) | etcd, encrypted | Good — etcd dump is safe |
| External store | Vault / cloud manager | Strongest — out of etcd, rotated, audited |
Native Secret (+ encryption at rest) — simple, built-in; with KMS encryption, safe at rest. Good for most cases.
External store — secret lives in Vault or a cloud manager, synced or mounted in. Adds rotation and audit; best for high-value credentials.
- Assuming Secrets are encrypted — without encryption at rest, etcd holds them in plaintext.
- Leaving broad
get secretsRBAC so any workload can read every credential. - Committing Secret manifests to Git in plaintext.
- Putting secrets in environment variables, where they leak into logs and child processes.
- Never rotating credentials, so a single leak is valid indefinitely.
- Enable encryption at rest with a KMS provider so an etcd dump does not expose secrets.
- Scope
get/list/watchon Secrets tightly and audit who holds those verbs. - Use an external secret store (Vault, cloud manager) for high-value credentials, keeping them out of etcd.
- Mount secrets as files rather than environment variables to limit leakage.
- Rotate credentials regularly and prefer short-lived, federated identities over long-lived secrets.
Knowledge Check
Without encryption at rest, how are Secrets stored in etcd?
- Base64-encoded plaintext — readable by anyone who can read etcd
- Encrypted by default with a per-cluster key the API server manages
- Hashed with a one-way digest so the value can never be recovered
- Held only in the API server's memory and never written to disk
What does enabling encryption at rest with a KMS provider achieve?
- Secrets are encrypted before being written to etcd, so an etcd dump no longer exposes them
- Secrets become readable by every Pod in the cluster automatically once the KMS key is active
- It removes the need for RBAC rules guarding the Secret resource
- It converts every Secret object into an encrypted ConfigMap
Why use an external secret store like Vault for high-value credentials?
- The secret stays out of etcd, with rotation and audit, so a cluster compromise doesn't expose the source of truth
- It makes Secrets load faster by caching them closer to the Pod
- It removes the need for ServiceAccounts when fetching credentials
- It transparently encrypts the pod-to-pod traffic that a NetworkPolicy already governs across every namespace in the cluster
You got correct