Chapter Fourteen · Performance and Scale
Performance and Scale
The load test said two instances carry 4,500 requests a second and the forecast for the biggest event of the autumn is 6,000, with 20,000 buyers arriving in the same second at 19:00:00. Six topics turn the numbers of Chapters 12 and 13 into the plan for that night: a profile that finds the time in the encoder rather than the database, a third and fourth instance that did nothing until a sticky-session setting was deleted, a token bucket per buyer and a bound per path, a waiting room built from a sorted set, the CPU moved off the loop, and an honest last page about the day one service stops being enough.
Everything in this chapter starts from a number somebody measured. The run of Chapter 12 found the knee at 4,500 requests a second with api-01's CPU at 100 percent, and the dashboard of Chapter 13 says which gauge moves first when it happens again. Marek profiles one checkout and finds that the 90 milliseconds everyone had charged to the database were 25 milliseconds of queries and 65 of serialization, validation and log formatting; three fixes take it to 32 without touching a query. He adds a third and a fourth instance and watches them idle for a day at 5 percent each, because the load balancer still pinned every returning buyer to the host they had started on, a setting that outlived the in-memory sessions Chapter 5 removed by two chapters. The pool arithmetic of Chapter 6 is redone for sixteen loops and does not fit, and PgBouncer goes in front of the primary.
Then the chapter turns to the night itself. A token bucket in Redis, atomic in one Lua script, limits each buyer to 10 holds a minute and each scanner key to 50 in a burst, keyed on the identity whose fairness it protects and placed in the rings on the right side of the password hash; a bound of 20 checkouts per loop refuses the twenty-first at the door with a 503 that arrives in a millisecond instead of after a 5-second pool wait. A waiting room puts the 20,000 buyers in a sorted set by arrival time, admits 300 a second because 300 is what the hold path measured, and hands each one a 10-minute token that is the only key to the seat map, so that the seats are sold in order rather than by whoever's retry landed. The CPU that had been stalling the loop, the 100-millisecond hash, the 4-second render, the 40-millisecond parse, goes to a bounded thread pool, a process pool or the worker, and loop lag on the dashboard is how the next one is found.
No wound opens here and none closes; the three from Chapter 1 were closed by Chapters 6, 7 and 8, and this chapter is what keeps them closed at 6,000 requests a second. On Stagedoor's first waiting-room night the sorted set at 20,000 members cost nothing, the primary held at 40 percent, and the ceiling was Payrail's 100 charges a second, which nobody had load-tested because the fake had no limit. The last topic is the book's handoff: the four signals that justify splitting one service, the render pipeline as the safe first split along the stream that already exists, the data boundary as the cost to price before any other, and the two courses, Middleware Deep Dive and System Design, that begin where this one stops.