Garbage Collection

A garbage collector frees memory the program can no longer reach, so the programmer never writes a free and can never free anything too early. There are two ways to find what is unreachable. Count the references to every object, or periodically trace everything reachable from the program's roots. They put their costs in opposite places: counting pays a little on every assignment and frees at once, tracing pays nothing per assignment and pays in batches.

CPython uses both. Neither one, in CPython or anywhere else, stops a program from holding memory it will never use again, and that gap is where every leak in a collected language lives.

Reachable, Not Used

A collector asks one question about each object: does a chain of references lead to it from a root? The roots are the places a program can reach without following anything: local variables on each thread's stack, global variables, and the interpreter's own tables. An object with such a chain is live. An object without one is garbage.

No collector can ask the other question, whether the program will ever read the object again. An object that is reachable and never read again is live as far as any collector can know. A dictionary used as a cache that nobody ever trims is perfectly reachable, and it grows until the process runs out of memory.

Reference Counting

With reference counting, every object carries a count of the references to it, raised and lowered on every assignment, every argument passed and every insert into a container. When the count reaches zero, the object is freed on the spot. That makes freeing deterministic: a file object closes the moment its last reference goes, not at some later collection.

It has two costs. Every reference change is a write to memory, even in code that only reads the object. And it cannot free a cycle: two objects that point at each other keep each other's counts above zero forever, even when nothing else can reach either of them.

import gc
class Node: pass
a, b = Node(), Node()
a.other, b.other = b, a    # each points at the other
del a, b                   # unreachable, but both counts are still 1
gc.collect()               # the cycle collector frees them: returns 2

The listing makes two objects that refer to each other and then deletes both names. Nothing can reach the pair any more, but each still holds a reference to the other, so neither count falls to zero. On CPython 3.15, the pair stayed alive after the delete, and a call to the cycle collector found and freed both, reporting two objects collected.

Tracing

A tracing collector starts from the roots, marks everything it can reach by following references, and reclaims the rest. It reclaims either by sweeping the unmarked blocks back into free lists, or by copying the live objects somewhere compact and releasing the old space whole. The marking is a graph traversal, the same one Chapter 8 teaches.

Nothing is paid per assignment, and cycles are no problem: an unreachable cycle is never marked, so it is reclaimed with everything else. The work comes in batches instead, and a batch that stops the program while it runs is a pause.

Generations

Most objects die young: a request's temporaries, a loop's intermediate results. Collectors exploit this by splitting objects by age, tracing the young ones often and cheaply and the old ones rarely. Java's and .NET's collectors and V8, the JavaScript engine, all rely on it, and so does CPython, for cycles only.

CPython's cycle collector has used three generations for most of its history. CPython 3.14.0 shipped an incremental collector with two generations, young and old, to shorten its longest pauses. After reports of significant memory pressure in production, the 3.14.5 release and Python 3.15 returned to the three-generation collector of 3.13. On 3.15 the collector reports three generations with thresholds of 2,000, 10 and 10. This is the most version-sensitive fact in the chapter, and it is stated with its version for that reason.

Pauses

A collection that stops every thread adds its whole duration to whatever requests are in flight. So garbage collection shows up in tail latency, the 99th percentile, not in the average. Modern tracing collectors do most of their marking while the program keeps running: Go's collector and Java's ZGC keep pauses below a millisecond, while Java's G1 aims for 200 milliseconds by default.

Reference counting has pauses too, in a different place. Dropping the last reference to a dictionary of ten million objects frees all ten million right there, on the thread that dropped it. On CPython 3.15, deleting a dictionary of ten million string values took 0.19 seconds, inside whatever request happened to do it.

Forked Workers and Reachability Leaks

A Python service pays reference counting on every line and a cycle collection now and then. The count updates have a second cost in servers that fork worker processes. The workers share their parent's memory until one of them writes to a page, which then gets copied (Chapter 10). Merely reading an object updates its count, so reading shared data turns shared pages into private copies. The cycle collector does the same on a larger scale: every collection writes into the header of every object it examines, live ones included. Instagram described this in 2017, when disabling the cycle collector bought its Python fleet about 10% more capacity, and CPython 3.7 added a way to move long-lived objects out of the collector's reach before forking.

The leaks that remain are all reachability leaks. An unbounded cache. A module-level list of every request seen. A caching decorator on a method, which stores each instance as part of its cache key and so keeps every instance alive. On CPython 3.15, an instance whose method carried an unbounded least-recently-used cache stayed alive after its last name was deleted and a collection ran.