Topic 21

What State Is and Why It Exists

StateConcept

State is Terraform's private map between the resources you declared in HCL and the real GCP resources it created. The google_storage_bucket.raw in your config maps to a specific bucket with a specific self-link, generation, and metadata, and state is where that mapping lives. It records, for every resource, the identifiers and attribute values Terraform last saw, so the next run knows exactly which API object each block of HCL refers to.

Without it, Terraform would have no way to know that the hatch-events-raw it sees in Google Cloud is the same one your config means — or whether it created that bucket at all, versus someone clicking it together in the Console. State is the answer to a single question Terraform asks on every run: of everything in this project, what do I manage, and what did each thing look like last time I touched it.

The Map Between Config and Reality

A resource in your config is an abstraction — google_storage_bucket.raw is a name you chose. The real thing is a bucket with a globally-unique name, a self-link, a creation timestamp, and dozens of other attributes. State holds the binding between the two: the address you wrote and the concrete GCP object it produced, plus a snapshot of that object's attributes as of the last apply.

That binding is what makes plan fast and precise. Terraform does not re-discover your infrastructure from nothing each run; it reads state to learn which API objects your HCL refers to, then checks only those. The map is the difference between "update the bucket I created in 2024" and "go find a bucket somewhere that looks like this one."

Why Terraform Doesn't Just Query GCP

A reasonable first instinct is that Terraform should skip state and just ask Google Cloud what exists. The problem is the question has no clean answer. A model that rediscovers everything from scratch would have to ask "which of the millions of objects in this project are mine?" — and GCP offers no universal "tag equals managed-by-this-config" query that would let it sort yours from everyone else's.

State is the recorded answer to "what do I manage," written down explicitly rather than inferred. Terraform notes each resource it creates the moment it creates it, so the set of managed resources is always known exactly, not guessed from labels or naming conventions that a human might also have used by hand.

State as the Source of What I Manage

A resource that is in state is one Terraform will reconcile on every plan and can destroy on a future apply. A resource that is not in state is invisible to Terraform — it will not touch it, will not reconcile it, and does not even know it exists, even if your HCL would happily create an identical one beside it.

This is the mechanism behind one of the most common surprises in Terraform. Delete a resource block from your HCL and Terraform reads that as "this is no longer managed, so destroy it" — because the block is gone but the entry is still in state. To stop managing something without deleting it, you remove it from state, not from config. Get this backwards and a refactor turns into an outage.

The Plan Is a Three-Way Diff

A plan is not a two-way comparison. Terraform reads three inputs: the config (the desired state in your HCL), the prior state (what Terraform last recorded), and a refresh of the real GCP resources (their current attributes, read live). The diff across all three is what produces the + create, ~ update, and - destroy lines, and the drift warnings when reality has moved out from under state.

Drift only surfaces because the refresh exists. If someone changed the hatch-events-raw bucket's storage class in the Console, the refresh sees the new value, state still holds the old one, and the config wants its own — Terraform shows you all three positions and proposes bringing reality back to the config. Reading that three-way diff is the single most useful habit in operating Terraform.

The three inputs to every plan

Config

The HCL you wrote — the desired state Terraform should converge toward.

Prior state

What Terraform recorded last apply — the last-known attributes.

Refresh

A live read of real GCP — the current attributes, where drift shows up.

Inspecting what state manages

# list every resource address Terraform is tracking
terraform state list

# show the recorded attributes for one resource
terraform state show google_storage_bucket.raw

These two commands read directly from state and are how you answer "what does Terraform think it owns?" without trusting the Console. state list prints every managed address; state show prints the attributes Terraform last recorded for one of them, which is often the fastest way to see why a plan proposes a change.

Performance and Refresh

State doubles as a cache. Because it stores the attributes Terraform last read, Terraform can compute much of a plan without re-fetching everything from GCP, which matters once a configuration manages hundreds of resources and a full refresh means hundreds of API calls. The cache is why a plan on a large state is not instant but is far faster than rediscovery from scratch would be.

There are two escape hatches when even that is too slow. -refresh=false trades freshness for speed by skipping the live read and trusting the cached attributes, and -target narrows the reconcile to a single resource or module. Both are deliberate exceptions for large or urgent situations — not defaults — and using them routinely means your plans stop reflecting reality.

In State vs Not in State

In state — Terraform reconciles the resource on every plan, can update it to match config, and can destroy it on a future apply. Removing its HCL block tells Terraform to destroy it.

Not in state — the resource is invisible to Terraform. It will not touch, reconcile, or destroy it, even if the same HCL would create an identical one. Adopting it requires an explicit import.

Common Mistakes

Deleting a resource by hand in the Console and expecting Terraform to forget it — state still lists hatch-events-raw, so the next plan shows + create to rebuild exactly what you just removed.
Removing a resource block from the HCL to "stop managing" something, not realizing that tells Terraform to destroy it on the next apply — to stop managing without deleting, you remove it from state with state rm, not from config.
Treating state as a disposable cache you can delete and regenerate — deleting state does not delete GCP resources, it orphans them, and the next apply tries to create duplicates that collide on globally-unique bucket names.
Hand-editing the state file to "fix" a value — there are real commands (state mv, import, state rm) for every legitimate change, and a hand-edit that desyncs from GCP produces a plan that fights reality on every run.
Running -refresh=false or -target by habit to make plans faster — your plan stops reflecting real GCP, drift goes unseen, and you apply against a stale picture.
Assuming two engineers can each keep a local copy of state — the moment one applies, the other's copy is behind and a plan from it proposes undoing work that already shipped.

Best Practices

Treat state as the authoritative record of what Terraform owns, and make every change to managed resources through Terraform so config, state, and GCP stay in sync.
Use terraform state list and terraform state show <addr> to inspect what is managed rather than guessing from the Console.
Never delete or hand-edit the state file to solve a problem — reach for import, state mv, or state rm, which keep the map consistent.
Run terraform plan before every apply and read the three-way diff between config, state, and a live refresh, because drift only shows up there.
Reserve -refresh=false and -target for genuine large-state or emergency cases, and return to full plans as the default once the pressure passes.

Comparable tools Pulumi same state model, different backend storage Config Connector no separate state — live cluster objects are the record gcloud stateless, which is exactly why it cannot diff

Knowledge Check

Why does Terraform keep a state file instead of querying GCP for what exists on every run?

GCP has no universal "this object is managed by this config" query, so state is the explicit record of what Terraform owns
Querying GCP live is fundamentally impossible because the resource APIs are strictly write-only and never return any attribute data once an object has been created
State exists purely to make the plan and apply CLI output read more prettily
GCP charges per API call, so caching in state lets Terraform avoid all live reads

You delete the hatch-events-raw bucket by hand in the Console. What does the next terraform plan show?

+ create, because state still lists the bucket and the refresh finds it gone, so Terraform wants to rebuild it
No changes, because Terraform notices the manual deletion and quietly drops the resource from state
- destroy, to reconcile the now-empty project with what state still records by clearing the leftover entry for the deleted bucket
An error that refuses to plan at all until you recreate the bucket by hand first

What are the three inputs Terraform compares to produce a plan?

The config (desired), the prior state (last-known), and a live refresh of real GCP (current)
The config, the dependency lock file that pins versions, and the pinned provider version metadata recorded alongside it
The saved plan file, the previous apply log, and the configured backend itself
The current state, the previous state, and the computed next state it derives

You want Terraform to stop managing a bucket but keep the bucket itself. What is the correct move?

Remove it from state with terraform state rm, leaving the GCP resource untouched and unmanaged
Delete the resource block from the HCL, which leaves the live bucket in place and simply stops Terraform from tracking it any further
Delete the whole state file so that Terraform forgets every resource it was tracking
Set the -refresh=false flag on the next apply so the bucket is left alone

You got correct