Not in the Request

The buyer needs to know the order is placed. She does not need to wait for the PDF to render, the email to send, the organizer's sales counter to update or the analytics event to be recorded, and on the night of the spring on-sale she waited for all four. POST /orders rendered the tickets inside the handler, 4 seconds of CPU on the event loop per order, and every other request on the instance waited behind it; by the end of the on-sale the queue of requests stretched to 40 minutes, and the emails arrived when the queue did. This is the third wound of Chapter 1, and this chapter closes it.

One sentence covers it: the request handler does the minimum that must be atomic and durable, commits, answers, and hands everything else to the worker through the outbox. The discipline is harder than the rule, because the temptation to do just one more thing inside the request arrives with every feature, and each one is small on its own. After the split, the handler's own work in place_order takes 90 milliseconds instead of 4 seconds, the buyer's wait is that plus Payrail's 400 milliseconds and nothing else, and the four things she did not need to wait for happen within a second of the commit on a machine that is not serving her.

The Test

Would the buyer notice if this happened 5 seconds later? If not, it is a job. The order row, the ticket rows and the seat status must be committed in the request, because the response says "you have 14C and 14D" and that sentence is either true at the moment it is written or it is a lie. The PDF, the email, the counter and the analytics event have no such deadline: the buyer reads the confirmation on the screen, and an email 2 seconds later is the same email. The test sorts checkout's work into two piles in under a minute, and the piles are stable; nothing has moved between them since Marek first drew them.

In code, the split means the handler's two transactions hold the order, its tickets and one outbox row, and the render call disappears from the request path entirely. The charge stays in the request, between them, for a reason the last section of this topic spells out.

async def place_order(req, ctx):
    async with conn.transaction():                       # the order exists before the money moves
        order_id = await insert_order(conn, req, status="pending")

    charge = await payrail.charge(req, idempotency_key=f"order-{req.public_id}",
                                 timeout=ctx.remaining(default=3.0))    # the client must know this now

    async with conn.transaction():                       # what the charge decided, in one unit
        await mark_paid(conn, order_id, charge)
        codes = await insert_tickets(conn, order_id, req.hold_ids)     # the codes exist at commit
        await outbox.add(conn, "render_tickets", order_id=order_id,
                         request_id=ctx.request_id, trace=ctx.trace.id)  # a row, not a publish
    return Created(order_id, tickets=codes, tickets_ready=False)   # 201 after 90 ms of handler time
    # no render, no mail call, no counter, no analytics anywhere in this function

The handler commits the order as pending, charges the card, then opens a second transaction that marks the order paid, inserts its tickets and inserts one outbox row naming the render job. Two transactions rather than one, because Topic 32 of Chapter 6 forbids holding a connection across a call that leaves the process, and because the pending row is what the reconciliation of Chapter 10 looks for when the answer to the charge is lost. It then returns 201 with the order and the ticket codes. What it does not contain is the point: no render, no call to the mail provider, no counter, no analytics call. The outbox row carries the order id and the request's context ids from Topic 21 of Chapter 4, and the relay of Chapter 7 moves it to the stream within 100 milliseconds of the commit.

What Changes for the Client

The 201 carries the order and its tickets' codes, because the codes exist at commit: insert_tickets generates them, and the scanner at the door will read them whether or not a PDF was ever rendered. What does not exist at commit is the PDF, and the response says so with tickets_ready: false. The client either polls GET /orders/{public_id} until the flag flips, which it does within 5 seconds on a normal night, or shows "your tickets are on their way" and lets the email do the rest. Chapter 2 stated the rule the flag obeys: a status code is a promise about the world at the moment the response is written. 201 promises the order exists. It promises nothing about the PDF, so the body must not imply it.

When the request itself is the job, the answer is 202. The organizer's CSV export from Topic 16 of Chapter 3 wants every order of an event, which is 200 rows a step through a cursor for as long as it takes; the handler inserts an export row and an outbox row, and returns 202 with a Location of /exports/{id}. That resource reports status: running, then done with a download link, and the organizer's browser polls it every 2 seconds. A 201 here would be the lie Chapter 2 warned about: a Location that points at a file which does not exist yet.

Through the Outbox, Not Directly

The handler does not call enqueue(). It inserts the outbox row inside its transaction, and the relay publishes. Chapter 7 derived why, and this topic says it once more because this is where the temptation lives: redis.xadd is one line, and the outbox is a table, a relay and 100 milliseconds of latency. The one line is wrong in a way no test catches. Publish before the commit, and the worker claims the job within 50 milliseconds and renders tickets for an order whose commit then fails on the seat constraint. Commit and then publish, and the deploy of Chapter 11 kills the process between the two lines, leaving an order that is durable and silent forever.

The outbox makes the order and its consequences one unit. Both rows commit or neither does, and the relay's only job is to move a row that already exists. The price is the 100 milliseconds and an at-least-once duplicate when the relay crashes between publishing and marking, which Topic 45 makes the handler tolerate. Between a lost job and a duplicated one, the duplicate is the failure a handler can be written for, and it is the one the design chooses.

What the Worker Is

The worker is the same codebase started with a different command: stagedoor worker instead of stagedoor api. It imports the domain layer and the storage layer of Topic 19 of Chapter 4 and has no transport layer at all, because nothing in it receives HTTP. Its handler for render_tickets calls the same render_tickets(order) function that a test calls and that the api called in the spring, with the same repositories underneath, the same mail client injected the same way, and the same domain errors raised. A rule fixed in the domain is fixed for both processes in the same commit, which is the first dividend the layering paid, and it paid it here.

What the worker adds is a consumer loop, which is Topic 44, a thread pool of 8 for the CPU-bound handlers so that a 4-second render never sits on its event loop, and its own connection pool, counted inside the 40 connections that Chapter 1 left after the api's 160. It runs on worker-01, and Topic 47 is about the night when one of it was not enough.

Jobs That Are Not Fire-and-Forget

The buyer does eventually need the email. A job that fails silently is a ticket that never arrives, and the support ticket about it arrives on the night, from a buyer standing at the door. "Later" has to mean "soon and certainly," and three mechanisms make it mean that: a retry policy for each job kind, so that the mail provider's 503 is tried again 5 times with backoff; a dead-letter stream, so that a render that fails on a malformed seat label is kept with its error for a human instead of vanishing; and queue age on the dashboard, so that a backlog of 12,000 jobs is a page at 60 seconds and not a complaint at 40 minutes. Topics 46 and 47 build them.

They exist before the job ships, not after its first failure. Stagedoor's rule for a new job kind is that its definition names the retry policy and the dead-letter destination, and the queue-age alert covers it by its kind label from the first deploy. A job without those three is the fire-and-forget the spring on-sale already ran, moved to another process.

When the Request Must Wait

Payrail's charge is not a job. The buyer must know whether the card was accepted before the response, because "your order is placed" followed 10 seconds later by an email saying the card was declined is a worse night than a slow checkout. The test here is a second question, asked alongside the first: must the client know the outcome now? The charge is the one thing in checkout that answers yes, so it stays in the request, with its 3-second timeout and its idempotency key from Chapter 7, and its 400 milliseconds at P50 are the floor of the buyer's wait.

The refund is the counterexample that shows the second question is different from the first. The buyer would notice the money 5 seconds late, which fails the first test, but she accepts "refund requested" now and the money in 3 to 5 days, because that is what refunds are. The Payrail refund call is a job, the response is 202, and the order's status moves from paid to refunded when the worker hears back. Two questions, and a piece of work goes in the request only when the answer to the second is yes.