Topic 48

What to Cache and What Not To

Caching

A cache is a copy that is allowed to be wrong for a while, and the decision to cache something is a decision about how wrong and for how long. The seat map of a hot event is read 2,600 times a second during on-sale and changes 3 times a second; a copy of it that is 2 seconds old is the case a cache exists for. A buyer's own order is read once, by her, and must be exact; a copy of it is a support ticket. The two reads sit 40 lines apart in the same storage layer, and nothing about the code tells them apart. What tells them apart is two numbers and one sentence, written beside each read, and this topic is how Marek arrives at them.

The chapter that follows puts the seat map in Redis, invalidates it, breaks it in four different ways and repairs each one. None of that machinery is worth building for a read that happens 10 times a minute or for a read that must never be stale. The decision comes first, and it is arithmetic rather than habit: reads per write, cost per read, and the price of showing an old value to the person reading it.

The Ratio and the Cost

The seat map for event 8812 is one query joining 2,000 rows of seats with their current holds, and it takes 8 milliseconds of the primary's CPU. At 2,600 reads a second that is 20.8 CPU-seconds every second: the primary is spending more than 20 cores' worth of work answering the same question. The map changes 3 times a second, when a hold is placed, expires or converts to a sale. So the ratio is roughly 870 reads per write, and every read past the first one after each write is recomputing a result that has not changed.

A copy that is rebuilt once per write and served from memory the other 869 times turns 2,600 queries a second into 3. That is 99.9 percent of the seat map's cost removed from the primary, and since the seat map was 2,600 of the 3,000 requests a second at peak, the primary's total load on the night the cache went in dropped by 90 percent. The cache is justified by that arithmetic and by nothing else. The same arithmetic on the buyer's order page, read perhaps twice per order and costing 2 milliseconds, says the cache would save 4 milliseconds per order and cost a Redis round trip on every read, and the answer there is no.

The seat map, before and after a copy that is allowed to be 2 seconds old

Readsper second at on-sale

2,600 requests for one map. 8 ms of primary CPU each, 20.8 CPU-seconds a second.

Writesper second at on-sale

3 changes: holds placed, holds expiring, seats sold. About 870 reads per write.

With the copyrebuilt once per write

3 queries a second on the primary instead of 2,600. 99.9 percent of the map's cost gone.

The pricewhat a buyer can see

A seat shown available up to 2 seconds after it was held. One 409, then another seat.

The Cost of Being Wrong

The second number is what happens when the copy is stale, and it is a number about people, not CPUs. A buyer sees seat 14C as available 2 seconds after another buyer held it. She clicks, the hold path of Chapter 6 takes the row lock, finds the seat held, and answers 409 with a Problem Details body naming the seat. She picks 14D. That is annoying, correct and acceptable: the truth was never in doubt, only the picture of it was late, and the write path refused to act on the picture. Every read that fails this way, where the stale value leads to a refused write rather than a wrong one, is a candidate for caching.

An order shown as pending 30 seconds after it was paid is a different kind of wrong. The buyer emails support, the support agent looks at the same cached page, sees an unpaid order and a receipt from Payrail that does not seem to match it, and charges her again to be safe. Nobody refused anything; a person acted on the stale value and money moved. A read whose stale value can be acted on, by a buyer, an agent or a job, is not cached, whatever its ratio. Topic 36 of Chapter 6 wrote the same classification for the replica, and this is that list again with the same answers: the reader decides, not the table.

What Stagedoor Caches

Five things, each with a stated staleness. The seat map, 30 seconds, and in practice milliseconds, because the write path deletes the key at commit and the 30 seconds is the floor for the day the delete is missed. The public event list, 60 seconds; an event appearing on the home page a minute after the organizer publishes it is unnoticeable. The organizer's sales totals, 60 seconds, on a chart that says when it last refreshed. The count endpoint of Topic 16, 60 seconds, for the one dashboard widget that wants a total instead of a next page. And sessions, from Topic 25, which are not a cache of anything: Redis is where they live, and the price of losing them is a sign-in, which Chapter 7's dependency table recorded as a decision.

What it does not cache is the list a support agent would recognize: orders, holds, tickets, payments, and anything else the buyer must see exactly as the database has it. The buyer's own held seats are on that list, which is why the seat map a buyer sees is two requests and not one: the public map, cached and identical for everyone, and her own holds, fetched separately and never cached. Chapter 2 made that split so the map could carry one validator for every client; here it is the split between a value that may be stale and one that may not.

Whether a read gets a copy, decided beside the read

Hundreds of reads per write, milliseconds of CPU each, and a stale value leads to a refused write?→Cache it; write the TTL beside the read

A person or a job can act on the stale value: money, a ticket, a refund?→Never; the primary, every time

Read 10 times a minute, or cheaper than a Redis round trip?→No copy; the arithmetic says no

Differs per user, like her own holds inside the public map?→Split it: cache the shared part, fetch hers separately

Lives only in Redis: a session, a lock, a rate-limit counter?→Not a cache; a decision about what its loss costs

Where the Cache Lives

In Redis, on redis-01, shared by api-01, api-02 and the worker. One copy, invalidated once, seen the same way by every process. The alternative is a dictionary in each process, which costs no round trip and no serialization, and it is wrong for any value that changes, because there are now as many copies as there are processes and no way to invalidate all of them. Stagedoor's first cache was that dictionary. A hold committed on api-01 deleted api-01's copy; api-02 kept serving the old map for the rest of its TTL, and the load balancer sent every buyer to both. Two instances, two truths, and the seat that looked available depended on which instance you were routed to.

Process memory has a place: values that never change while the process runs. The parsed configuration of Topic 23, the compiled route table, the list of ticket sections, the public key that verifies tokens. Those are loaded once at start and are not a cache, because there is no fresher version to be out of date against. The test for whether a value may live in process memory is whether a write anywhere in the system could make it wrong; if yes, it goes to Redis, where there is one of it.

The Cache Is Not the Truth

Every value in Stagedoor's Redis is one of two things: a copy of something Postgres has, or coordination state whose loss is a known event. The seat map is a copy; the next read rebuilds it from the primary in 8 milliseconds. The rate-limit counters are coordination state; losing them means 30 seconds of unlimited scanning until they refill. Sessions are coordination state; losing them means a sign-in. Nothing in Redis is the only place a business fact lives, and the way to check is to ask what a flush would cost. For Stagedoor the answer is a minute of slower seat maps and some signed-out browsers, which is to say time, and nothing that anyone would have to reconstruct by hand.

The design that fails this test is the one Topic 52 catalogues: a counter incremented in Redis and nowhere else, the number of seats remaining for an event, which is right until the first restart and then zero. A cache that holds something the database does not is a second database, with none of the durability and none of the backups, and the moment it holds one such thing the flush stops being a non-event and becomes an incident. Stagedoor keeps the rule mechanical: a value goes into Redis only if the code that reads it has a path to rebuild it or a written answer for what its absence means.

Measuring

Three numbers on the dashboard per key family: the hit ratio, the primary's query rate for that read, and the staleness actually observed. The hit ratio is hits over hits plus misses, counted in the storage layer where the GET happens, and it says whether the cache is doing anything. A key family at 40 percent is paying a Redis round trip on every request and a database query on 60 percent of them, which is worse than no cache at all; the seat map at on-sale sits above 99 percent and the event list above 95. The primary's query rate for the seat map is the same number from the other side, and it is the one that shows the night the invalidation broke and the rate went from 3 a second to 300.

Staleness is measured, not assumed. Every cached value carries the time it was built, and the read path reports the age of what it served, so the dashboard shows that the seat map is served at a median age of 400 milliseconds and a maximum of 30 seconds, and a family whose values are always served under a second with a 30-second TTL is telling you the TTL could be 5 minutes at no cost. The reverse is the more common finding: a TTL of 30 seconds on a value that changes twice a day is spending a rebuild every 30 seconds to be fresh about nothing. The numbers say which; the next topic builds the code that produces them.

Common Mistakes

Caching the buyer's order — the paid order shows as pending for 30 seconds, the support agent reads the same stale page, and charges her a second time to be safe.
An in-process dictionary for shared data — two instances hold two different seat maps, only the one that took the write invalidates, and which seats look available depends on which instance the load balancer picked.
Caching before measuring — a Redis round trip and a serializer added to a read that happens 10 times a minute, for a saving nobody can see on any graph.
The cache as the store — the seats-remaining counter that was incremented in Redis and nowhere else, and was zero after the restart until an organizer noticed.
No hit-ratio metric — a key family that has been all misses since the key format changed, costing a round trip on every request, and nobody knows because nothing counts it.

Best Practices

Cache by arithmetic: reads per write times cost per read, against an acceptable staleness written in a comment beside the read.
Put shared data in Redis and only values that never change while the process runs in process memory.
Make every cached value rebuildable from Postgres, and treat a Redis flush as a minute of slower pages rather than an incident.
Put the hit ratio, the primary's query rate and the observed staleness on the dashboard per key family from the first day the cache exists.
Split any response that mixes a shared value with a per-user one into two reads, so the shared part can be cached and the personal part never is.

Comparable toolsRedis and Valkey the shared cache, one copy for every instanceMemcached the older shared cache, values only, no persistenceDjango cache framework, Spring Cache, Rails.cache the framework layers over the same decisionCDNs the HTTP layer of the same idea, Topic 51

Knowledge Check

Marek is deciding whether a read deserves a cache. Which two numbers make the case?

Reads per write and the CPU cost of one read
Redis's free memory and the size of the pool
The size of the value and its compression ratio
The number of instances and the chosen TTL

The buyer's order page has a read ratio of 2 reads per write and costs 2 ms on the primary. Why is it excluded from the cache, whatever its numbers?

Because the page differs per user and a shared cache cannot hold one copy per buyer
Because orders change too often for any TTL to keep the cached copy close to the truth
Because a person can act on a stale copy of it, and the stale action moves money
Because the order's shape does not serialize into the JSON format the cache stores

Stagedoor's first seat-map cache was a dictionary in each api process. What went wrong?

Each lookup was slower than a Redis round trip because the dictionary was unindexed
A hold on api-01 cleared only api-01's copy, and api-02 served the stale map until its TTL
The two instances overwrote each other's entries because they shared the same memory
Each process ran out of memory holding 2,000 seat maps and restarted under the load balancer

Someone runs FLUSHALL on redis-01 by mistake during an ordinary afternoon. In a correctly designed Stagedoor, what does it cost?

Every pending job in the stream is gone for good and the seat map must be restored from a backup
Every hold placed in the last 10 minutes is lost and the seats return to available for other buyers
Every browser session survives, but the worker loses its tickets and the buyer has to re-order
A minute of slower seat maps, signed-out browsers, and rate limits that refill; nothing to reconstruct

You got correct