Topic 28

Multi-Stage Builds

BuildLayers

A multi-stage build writes several FROM stages in one Dockerfile: an early builder stage that has the compilers, headers, and package caches needed to produce artifacts, and a final stage that starts from a slim base and uses COPY --from=builder to take only the finished artifacts across. The build toolchain stays in the discarded builder stage and never reaches the shipped image.

This is the single change that takes the naive Driftwood image from 1.1 GB to roughly 180 MB, and it is the spine the rest of this chapter hangs off. Everything else — the lean context, the cache and secret mounts, the deliberate shrinking — refines an image whose shape this topic sets.

The Two-Stage Shape

FROM python:3.12 AS builder starts a stage that has the full toolchain: it installs gcc and headers, then compiles Driftwood's dependencies into wheels. FROM python:3.12-slim then begins a second, separate stage that copies only those built wheels across and installs them. gcc, the headers, and the intermediate object files live in builder and are thrown away when the build ends — only the second stage becomes the image.

Driftwood's multi-stage Dockerfile — builder compiles wheels, slim runtime ships them

# syntax=docker/dockerfile:1
FROM python:3.12 AS builder
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends gcc python3-dev libpq-dev
COPY requirements.txt .
RUN pip wheel --no-deps --wheel-dir /app/wheels -r requirements.txt

FROM python:3.12-slim
WORKDIR /app
COPY --from=builder /app/wheels /wheels
RUN pip install --no-index --find-links=/wheels /wheels/*
COPY . .
RUN useradd --create-home app
USER app
EXPOSE 8000
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "driftwood.wsgi"]

The first stage carries gcc, python3-dev, and libpq-dev only so pip wheel can compile the C-backed dependencies into /app/wheels. The second stage is python:3.12-slim with no compiler at all; it installs from the prebuilt wheels with --no-index, copies the app, drops to a non-root user, and sets the gunicorn CMD on :8000.

COPY --from — The Hand-off

The final stage pulls named files out of an earlier stage with COPY --from=builder /app/wheels /wheels. That copy is the only path between stages — nothing else from the builder crosses. Its apt cache, its gcc, its intermediate object files, the libpq-dev headers: none of them exist in the result. You take the artifacts you want by name and leave the rest behind in a stage that is discarded.

This explicitness is the whole mechanism. There is no implicit inheritance from one stage to the next — a runtime stage starts from its own FROM with an empty slate, and a file only appears in it if a COPY --from brought it there.

Why the Toolchain Must Not Ship

The build needs gcc, python3-dev, and the libpq headers to compile C-backed wheels like psycopg2. The running gunicorn process needs none of them — it executes already-compiled code. Shipping the toolchain anyway adds roughly 900 MB to the image, dozens of extra packages to patch on every CVE cycle, and a working compiler that an attacker who lands in the container would otherwise have to bring themselves. The builder stage exists precisely so all of that stays out of production.

What each stage carries — and what reaches production

builder stage (discarded)

python:3.12 + gcc, headers, apt cache, intermediate objects, the compiled wheels. Roughly 1 GB. None of it ships except the wheels.

runtime stage (shipped)

python:3.12-slim + the installed wheels + the app, running as a non-root user. Roughly 180 MB. No compiler present.

Stages Are Cheap and Composable

A multi-stage build is not limited to two stages. You can add a deps stage, a test stage, and a runtime stage, target any one of them with docker build --target builder when you need to debug it directly, and let BuildKit build independent stages in parallel (topic 27). Only the final stage — or the one named by --target — is kept; every other stage is build-time scaffolding that the result never carries.

The Driftwood Rebuild, Concretely

Put together, the rebuild is direct. The builder stage compiles the wheels into /app/wheels; the runtime stage starts from python:3.12-slim, pip installs those prebuilt wheels with no compiler present, copies the app, and sets the gunicorn CMD on :8000. The result drops from 1.1 GB to roughly 180 MB, carrying only the Python runtime, the installed dependencies, and the app.

Same app, same behavior, same port — about a sixth of the size. The 900 MB difference is the toolchain that stayed in the builder stage, and topic 31 will measure exactly where it went using docker history.

Multi-Stage vs Single-Stage

Single-stage build — one FROM, everything in the same image: install the toolchain, compile, and run. Simple to read, but the compiler and build caches ship to production, inflating both size and attack surface. Reach for it only for a trivial image with no build step.

Multi-stage build — compile in a throwaway builder stage and COPY --from only the artifacts into a slim final stage. A little more Dockerfile, but the shipped image carries only what runs. Reach for it the moment compilation or asset building is involved — which, for Driftwood, is the difference between 1.1 GB and 180 MB.

Common Mistakes

Installing the build toolchain in the final stage "to be safe" — it defeats the entire point; gcc and the headers belong only in the builder stage that gets discarded, never in the image that ships.
Writing COPY --from=builder / / or copying the builder's whole filesystem into the final stage — you have just re-shipped the toolchain you went to the trouble of isolating, and the image is back to 1.1 GB.
Forgetting that each stage starts from its own FROM with nothing carried over implicitly — files cross only via explicit COPY --from, so a runtime stage that expects /app to already exist because the builder created it will fail.
Naming stages inconsistently, or referencing them by numeric index, so a COPY --from repoints to the wrong stage after someone reorders the file — name every stage with AS and reference it by that name.
Compiling wheels in a python:3.12 (glibc) builder and installing them in an alpine (musl) final stage — the prebuilt wheels are ABI-incompatible and break at import time; keep the libc family consistent across stages.

Best Practices

Split every image with a compile or asset-build step into a builder stage and a slim runtime stage, copying only the artifacts across with COPY --from.
Name each stage with AS <name> and reference it by name in COPY --from, so reordering the stages can never silently repoint a copy.
Keep the builder and runtime stages on the same libc family — both glibc, or both musl — so compiled wheels and binaries stay ABI-compatible across the hand-off.
Use --target to build and debug an intermediate stage directly when a compile fails, rather than tearing the multi-stage structure apart to investigate.

Comparable tools Buildpacks · ko reach the same toolchain-free result with no hand-written multi-stage Dockerfile Kaniko · Podman · Buildah execute the same multi-stage Dockerfile syntax BuildKit · buildx builds the independent stages in parallel

Knowledge Check

What does a multi-stage build keep, and what does it discard?

It keeps the final stage and the artifacts copied into it; the builder stage with its toolchain and caches is discarded
It merges every stage's filesystem into one combined squashed image
It keeps the builder stage's compiler and headers so the image can rebuild itself later
It ships the full builder stage with its compiler and apt cache intact and discards the slim runtime stage along with the copied artifacts

Why must the build toolchain stay out of the final image?

The running process never uses gcc or headers; shipping them adds ~900 MB, more to patch, and a compiler an attacker could exploit
Keeping the compiler in the image makes the application start up faster at runtime
The application cannot run at all without the compiler and headers present at runtime
Docker refuses to push any image larger than 1 GB to a remote registry, so the toolchain must be stripped to stay under the hard upload limit

How does a file get from the builder stage into the final image?

Only via an explicit COPY --from=builder; nothing else from the builder is inherited
Everything the builder created is automatically available in the final stage by default
The two stages share a mounted volume that carries files between them during the build
The builder pushes its files to a registry that the runtime stage pulls

Why can compiling wheels in a python:3.12 builder and installing them in an alpine final stage break the image?

The wheels are compiled against glibc but alpine uses musl, so they are ABI-incompatible and fail at import time
alpine runs on a different CPU architecture than the python:3.12 builder image, so the wheels are compiled for the wrong instruction set and segfault on load
Docker forbids COPY --from between two stages built on different base images
alpine ships no Python interpreter at all, so the copied wheels cannot install

You got correct