Layer Caching and Instruction Order

The build cache is the difference between a 90-second rebuild and a 2-second one, and it turns entirely on the order of your instructions. The daemon caches the result of each instruction and reuses it on the next build until it hits the first instruction whose inputs changed — and from that point down, every layer is rebuilt, cache invalidation cascading to the bottom.

The whole discipline follows from that one rule: order your instructions so the things that change rarely sit above the things that change on every commit, and the expensive steps stay cached. For Driftwood that is the difference between reinstalling every dependency on each one-line code edit and reinstalling none of them. This topic is the spine the rest of the chapter hangs on.

How the Cache Decides

For most instructions the daemon hashes the instruction text. For COPY and ADD it hashes the instruction text plus the contents of the files being copied. An unchanged hash means the cached layer is reused; a changed hash invalidates that layer — and every layer below it. The subtlety that catches people is the file-contents part: editing a file that a COPY brings in changes that instruction's hash even though the instruction text is byte-for-byte identical.

The Cascade

Cache invalidation flows downward only. Change instruction 4 and instructions 4 through the end rebuild, while 1 through 3 stay cached. Nothing above a change is ever affected — which is the entire basis of ordering. Put the volatile instruction near the bottom and a change to it rebuilds little; put it near the top and a change rebuilds almost everything beneath it.

The COPY . . Before RUN install Trap

Driftwood's naive Dockerfile copies the whole source tree and then installs dependencies. Because COPY . . includes the application code, editing a single line of Python changes the copied files, busts that layer, and forces a full pip install on every rebuild. That is the 90-second case — reinstalling every dependency because one line of unrelated code moved.

FROM python:3.12
WORKDIR /app
COPY . .                              # any code edit busts this layer…
RUN pip install -r requirements.txt   # …forcing a full reinstall here
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "app:app"]

The RUN pip install sits below the COPY . ., so the cascade rule guarantees it rebuilds whenever the copy does — and the copy changes on every code edit. The dependencies have not changed at all, but the cache has no way to know that, because the layer that installs them depends on the layer that copied everything.

The Copy-Deps-First Fix

Copy requirements.txt alone, run pip install, then COPY . .. Now an application code change invalidates only the final copy layer; the cached dependency layer above it is reused untouched, and the rebuild drops to roughly 2 seconds. Dependencies reinstall only when requirements.txt itself changes — which is the behavior you actually wanted.

FROM python:3.12
WORKDIR /app
COPY requirements.txt .               # changes only when deps change
RUN pip install -r requirements.txt   # cached across code edits
COPY . .                              # the only layer a code edit busts
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "app:app"]

Two lines moved, and the rebuild went from 90 seconds to 2. This is the lesson the whole chapter is built around: the same instructions, reordered least-to-most volatile, change the economics of every rebuild for the life of the project. The same pattern applies to package.json before npm install, go.mod before go build, and every other manifest-then-code stack.

What Else Busts the Cache

A changed RUN apt-get update mid-build, a moving base tag like python:latest that resolves to a new digest, an ARG value passed with --build-arg, and an ADD of a URL all invalidate from that point down. Understanding the inputs to each instruction's hash — instruction text, copied file contents, build args — is how you predict a cache miss before it costs you, rather than accepting "builds are just slow."