Chapter 10: Structuring Projects & Environments at Scale
Topic 67

DRY Configuration Patterns

Patterns

The standing objection to directory-per-environment is duplication. If envs/hatch-app-prod/ and envs/hatch-app-staging/ are separate directories, won't they drift apart the moment someone fixes one and forgets the other? It is a fair worry, and the answer is that native Terraform already has the tools to keep environment directories thin and identical in structure, differing only in values — no wrapper tool required.

Three patterns do the work: shared child modules so the logic lives once, per-environment .tfvars so only values differ, and partial backend config so the backend block is one template. Together they make a directory split DRY without giving up the physical isolation the previous topic argued for. This topic also marks the ceiling — the point where chasing DRY makes the code worse, not better.

Shared Child Modules

The real logic lives once, in modules/service-project or modules/ingest-pipeline. Each environment directory is a short module call into it. Changing the module changes every environment on the next apply, and there is nothing to copy, because the directories never held the logic in the first place — only the call to it. This is the same modules/ + envs/ split from Topic 64, now seen as the primary DRY mechanism.

The discipline is that no resource block belongs in an envs/ directory. The moment a real resource appears there instead of a module call, it is a candidate to drift, because its twin in the other environment is a separate copy. Push it into the module and the surface where two environments can diverge shrinks to the variables you pass in.

Two ways to keep environments alike
Shared module + .tfvars
Logic lives once; environments differ by a readable values diff. Stays DRY, a fix reaches all.
Copied main.tf per env
Full config duplicated and edited in place; the next fix lands in one copy and the rest drift.

Per-Environment .tfvars

The only thing that genuinely differs between prod and staging is values — a machine size, a replica count, a project ID. Keep those in prod.tfvars and staging.tfvars and pass them with -var-file. The .tf files stay structurally identical across environments; the difference between prod and staging is one file of values, which reads as a diff and reviews as a diff.

Identical structure, divergent values
# envs/hatch-app-prod/prod.tfvars
project_id     = "hatch-app-prod"
min_instances  = 3
db_tier        = "db-custom-4-16384"

# envs/hatch-app-staging/staging.tfvars
project_id     = "hatch-app-staging"
min_instances  = 0
db_tier        = "db-f1-micro"

Both directories run the same main.tf calling the same module; the apply differs only by -var-file. You can read the entire production-versus-staging difference in those few lines, instead of diffing two full configurations to find what changed.

Partial Backend Config

The backend block can omit the per-environment prefix and supply it at init time. Leave the gcs backend with only the bucket, then run terraform init -backend-config="prefix=apps/hatch-app-prod". One backend template serves every stack, with one state key per environment, and there are no duplicated backend blocks where someone can paste prod's prefix into staging's directory.

Partial backend: prefix supplied at init
# backend.tf — identical in every env directory
terraform {
  backend "gcs" {
    bucket = "hatch-tfstate"
    # prefix intentionally omitted — supplied at init
  }
}

The init command carries the only per-environment fact the backend needs. Because the prefix is passed in rather than written in each directory, the backend block is genuinely the same file everywhere, and the surface where someone could point staging at prod's state shrinks to one CI parameter instead of a hardcoded line per directory.

Avoiding Copy-Paste Drift

The failure these patterns prevent is concrete: a fix applied to prod/main.tf but forgotten in staging/main.tf, so the environments quietly diverge in a way no one notices until staging stops predicting prod. Pushing all logic into modules/ removes the surface where the two can diverge at all — there is no main.tf resource block to fix in two places, only a module to fix once.

What is left after that is intended difference, and it lives in .tfvars where it is visible. Drift becomes a thing you choose, line by line, in a values file, rather than a thing that happens to you because two copies of the same resource fell out of sync. The structure carries the sameness; the .tfvars carries the difference.

Not Over-Abstracting

DRY has a ceiling. A module so parameterized that every value is a variable and the call site is unreadable is worse than a little duplication — you have traded copy-paste drift for a module nobody can reason about, where fifty inputs hide what an environment actually deploys. When the module's inputs outnumber its readers' patience, the abstraction has gone past the point of clarity.

Some genuine per-environment divergence belongs as explicit config, not another conditional flag. A prod-only disaster-recovery replica is clearer as a resource in the prod directory than as a create_dr_replica variable threaded through the module and defaulted off everywhere else. Stop abstracting when the next variable makes the module harder to read than the duplication it removes; a little honest repetition beats a parameter maze.

Shared module vs duplicated config

Shared module + .tfvars — logic lives once in modules/, environments are a module call plus a values file. A fix lands in one place and reaches every environment; the only difference between prod and staging is a readable diff of values. This is the default.

Duplicated config per directory — full main.tf copied between environments and edited in place. Fast to start, but the next fix lands in one directory and the environments silently diverge. Reserve raw duplication only for genuine one-off divergence that an over-parameterized module would obscure.

Common Mistakes
  • Copying full main.tf files between environment directories and editing values in place — the next fix lands in one directory and the environments silently diverge.
  • Hardcoding the backend prefix in each environment's backend block instead of supplying it with -backend-config, multiplying the surface where someone points staging at prod's state.
  • Encoding every conceivable difference as a module variable until the module has fifty inputs and no reader can tell what an environment actually deploys — abstraction past the point of clarity.
  • Storing secret values in committed .tfvars files instead of sourcing them from Secret Manager at apply time — the .tfvars ends up in git with credentials in it.
  • Leaving a real resource block in an envs/ directory instead of in the module — it now has a separate twin in every other environment, exactly the drift surface DRY was meant to close.
Best Practices
  • Put all resource logic in shared modules/ and reduce each env directory to a module call plus a .tfvars.
  • Differentiate environments only through -var-file per-environment .tfvars, keeping the .tf structure identical across environments.
  • Supply the per-environment backend prefix with -backend-config so one backend template serves every stack.
  • Source secrets from Secret Manager at apply time rather than committing them in .tfvars, so credentials never land in git.
  • Stop abstracting when the module's inputs outnumber its readers' patience — accept explicit config for genuine, rare per-environment divergence.
Comparable tools Terragrunt generates backend and provider blocks to enforce this DRY-ness automatically Pulumi config per-stack Pulumi.<stack>.yaml as the .tfvars analog Terramate code generation for shared backend config Kustomize the conceptual base-plus-overlay cousin

Knowledge Check

How do shared child modules eliminate copy-paste drift between environment directories?

  • The logic lives once in modules/ and each env is a module call, so there is no second copy of a resource to fix and forget
  • They mark the per-env directories read-only at the filesystem level so no engineer can ever edit one resource into a state that drifts from the others
  • They merge the prod and staging directories into one shared state file so the two can never diverge
  • They run a line-by-line diff between the env directories on every apply and block any divergence they find

What does partial backend config with -backend-config solve?

  • It lets one backend template omit the per-env prefix and supply it at init, so no directory hardcodes a state key to mispaste
  • It client-side encrypts the state file with your own customer-supplied key before the object is ever uploaded to the GCS bucket
  • It lets two environments safely share a single state prefix without their objects ever colliding
  • It removes the need for a state bucket entirely by keeping the state inline in the config

Why prefer per-environment .tfvars over duplicated .tf files?

  • The .tf structure stays identical and the whole prod-vs-staging difference is a small, readable values diff
  • .tfvars files are parsed and applied faster than the equivalent .tf resource definitions
  • Terraform reads variable values only from .tfvars files and never from the variable defaults or definitions declared in .tf
  • Duplicated .tf files across directories cannot each declare a remote GCS backend

When has DRY gone too far?

  • When a module has so many input variables that no reader can tell what an environment deploys — explicit config would read clearer
  • Whenever a shared module is called by more than one environment directory at the same time
  • As soon as any single value is passed through an input variable instead of being hardcoded directly inside the resource block where it is used
  • When two environments are pinned to the same shared provider version constraint

You got correct