The Service Behind the Screen
A backend is a process that listens on a port, accepts requests it did not schedule from clients it does not control, keeps the only copy of the truth in a store, and answers. That one sentence holds every hard problem in this book. The requests arrive at the same time, so the truth has to stay consistent while several of them touch it. The truth lives in one place, so every copy of it anywhere else is already slightly wrong. And the answer travels back over a network that can drop it after the work is done.
Stagedoor is one such process — an event-ticketing service with two API instances, one worker, one Postgres primary with a replica, one Redis, and a payment provider on the far side of the internet. By the end of this book you will know every part of it well enough to predict how it fails. This topic is the map: what each box is, which of them holds state, and which four verbs every request passes through.
What a Backend Is Not
It is not the database. The database keeps the truth; the backend decides what is true. When a buyer asks for seat 14C, Postgres does not know whether the hold is allowed — it knows the row says available. The rule that a hold lasts ten minutes, that a buyer may hold at most four seats, that a seat with an expired hold is available again: that is the service's job, and a system that pushes those rules into triggers and stored procedures has moved the backend into a place where the test suite, the code review and the trace cannot see it.
It is not the frontend either. The page renders and asks; it never decides. A seat drawn green because the page believes it is available is a guess, and two honest browsers guessing at once is exactly how 14C was sold twice. And it is not the infrastructure. The machine, the container, the orchestrator run the process; they do not know what an order is. The backend is the decision layer, and everything else in the picture is either an input to it or a place it keeps things.
Stagedoor in One Picture
Two api processes sit behind a load balancer on api-01 and api-02. A worker process on worker-01 runs the same code but takes its work from a queue instead of a socket. pg-primary holds the database and streams every change to pg-replica-a, which serves the organizer reports. redis-01 holds the seat-map cache, the job stream and the rate-limit counters. And Payrail, the payment provider, is a hostname on somebody else's network that answers in 400 milliseconds most of the time and in three seconds some of the time.
Every box in that picture is a separate failure domain, and every arrow between two boxes is a network hop that can time out, deliver twice, or lose the reply. Chapter 7 is about the arrows. For now it is enough to see that a single buyer's checkout crosses at least four of them before the tickets exist.
Request, Response, and the Thing in Between
A client sends bytes. The service parses them into a typed request, runs the domain logic, touches the store, and serializes a response. Four verbs: parse, decide, persist, answer. Chapter 3 is about parsing at the boundary so that nothing untyped gets inside. Chapters 4 and 5 are about deciding — the layers the logic lives in and the identity it decides on behalf of. Chapter 6 is persisting without losing a race. And answering, which sounds like the trivial part, is where the double charge of Chapter 7 begins, because a response that never arrives looks to the client exactly like work that never happened.
@router.post("/events/{event_id}/seats/{label}/hold") async def hold_seat(event_id: EventId, label: SeatLabel, principal: Principal, svc: Service) -> HoldOut: hold = await svc.holds.create(principal.user_id, event_id, label) # decide + persist return HoldOut.from_domain(hold) # answer
The parsing is invisible in that handler on purpose: by the time the function body runs, event_id is an EventId and label is a SeatLabel that already matched the seat-label pattern, or the request was rejected with a 422 before any of this code executed. The decision and the write happen inside svc.holds.create, in the domain layer, where the worker can call the same function without an HTTP request in sight. The handler's whole job is to translate between the outside vocabulary and the inside one — and that translation, done exactly once at the edge, is the second idea this book returns to on almost every page.
The Truth Lives in Exactly One Place
The seats row on pg-primary is the only authority on whether 14C is available. The seat map in Redis is a copy that may be thirty seconds old. The green square on the buyer's screen is a copy of that copy. The PDF ticket is a copy of the order. A service that lets two copies disagree about who owns a seat has already lost, and no amount of retrying fixes it, because there is no longer a fact to retry toward.
Keeping the one copy correct while 3,000 requests a second reach for it is Chapter 6's subject, and it is where the first of Stagedoor's three wounds is closed. Keeping the copies close enough to the truth to be useful is Chapter 9's. The rule that makes both chapters possible is stated here and never relaxed: Postgres holds the truth, Redis holds copies and coordination state, the client's token holds the caller's identity, and nothing else holds anything.
Stateless by Design
An api process remembers nothing between requests. That is not an accident of the framework; it is the property that lets api-02 answer a request api-01 started, lets the load balancer be dumb, and lets a deploy replace every process without anyone noticing. Every piece of state has a home outside the process, and the homes are the three named above.
Stagedoor's first version broke this rule in the most natural way possible: a dictionary of active holds, keyed by seat, living in the process's memory. It worked flawlessly with one instance. With two, a buyer's release request landed on the instance that had never seen the hold, the dictionary said there was nothing to release, and the seat stayed held for ten minutes while the buyer refreshed. Nothing in the logs was wrong. The state was simply in a place only one of the two processes could see, and that is the shape of every bug this rule prevents.
Reading the Codebase
One repository, one stagedoor package, three entry points. stagedoor api binds the port and serves HTTP. stagedoor worker consumes the job stream. stagedoor migrate applies schema changes and exits. The three share the domain and storage layers you will meet in Chapter 4 and differ only in how work arrives. This book quotes that codebase; it does not build it step by step, and a reader who writes Go or TypeScript should lose nothing but the syntax.
The machine is api-01: a host with a CPU count and a memory limit, replaceable, and owned by the infrastructure courses. When someone says the server is out of disk, they mean this.
The process is one running copy of the code — the unit that scales, crashes and restarts. There are always at least two, and when a page in this book says "the service," it means one of these.
The code is what this book is about. It is the same on every instance, it holds no state, and everything interesting it does happens on an arrow to somewhere else.
- Keeping state in the process — a dictionary of active holds works with one instance and silently corrupts with two, because the load balancer sends the release to the instance that never saw the hold.
- Treating the database as the backend — business rules written as triggers and stored procedures run where the test suite, the code review and the trace cannot see them, and the service no longer owns its own decisions.
- Letting the frontend decide — a seat shown as available because the page said so, with no check at the edge, is how two honest browsers sell 14C twice.
- Drawing the system without the arrows — the boxes are the easy part; every incident in this book lives on a hop between two of them, and a diagram that omits the hops omits the failure modes.
- Assuming one instance — code that reads "the current instance's" anything is code that will be wrong on the day the autoscaler adds a second one, which is a Tuesday, not a milestone.
- Draw the system with its failure domains before writing a handler, and write beside each arrow what the request does when the far side is unreachable — Chapter 7 turns that list into code.
- Give every piece of state exactly one home and name it: Postgres for the truth, Redis for copies and coordination, the token for identity.
- Keep the process stateless so that the instance count is a deployment setting rather than an architecture decision.
- Read a service by its entry points and its layers, not by its routes — the routes are the surface, the layers are the shape.
- Put the business rules in the service's own code, where a unit test can run them without a database, and leave the database to enforce constraints.
Knowledge Check
A buyer's screen shows seat 14C as available, the Redis seat map shows it as available, and the seats row in Postgres shows it as held. Which one is right, and why?
- The Postgres row, because it is the single authority and the other two are copies that lag
- The Redis seat map, because it is the copy the service actually reads on the hot path
- The buyer's screen, because it reflects the most recent response the service sent
- None of them, because three disagreeing copies mean the state is undefined until they match
Stagedoor kept active holds in a dictionary inside the api process. What broke, and when?
- Holds were lost on every single request, because the framework recreated the dictionary for each incoming request
- Releases failed once a second instance was added, because the hold lived in memory only one process could see
- Postgres rejected every second hold, because the row stayed locked by the dictionary's write until the process restarted
- The process ran out of memory during the on-sale, because expired holds were never evicted from the dictionary
Which of the four verbs — parse, decide, persist, answer — does the database own?
- Parse, because the database checks the type of every value before it agrees to store it in a column
- Decide, because business rules belong in triggers, right next to the data they govern and protect
- Persist, because keeping the truth durable and consistent under concurrent writes is the store's one job
- Answer, because the query result set is what the client ultimately receives in the body of the response
Why does the book insist that the api process holds nothing between requests?
- Because process memory is the most expensive tier of storage and every byte held raises the instance's monthly cost
- Because any instance must be able to serve any request, and a deploy must be able to replace any process
- Because Redis is faster than process memory for every kind of lookup once the working set exceeds a few megabytes
- Because the event loop cannot safely hold mutable state across await points without a lock around every access
You got correct