Parallelism and Its Bill

Running three lookups at once cuts wall-clock time and changes nothing else. Running three agents at once cuts wall-clock time and multiplies tokens, because each one carries its own prompt, its own tool schemas and its own growing context. Both get called parallelism in design documents, and only one of them is close to free.

The distinction matters because the cheap version is routinely left unused while the expensive one gets adopted on the strength of a diagram. At Sundry the free win was worth 0.9 seconds on every ticket that needed more than one lookup, and it took an afternoon.

Parallel Tool Calls Are Nearly Free

A model can request several tools in one turn, and when those requests are independent your dispatcher can run them concurrently. The token bill is identical either way — the same schemas went out, the same results come back — so the only thing that changes is how long the turn takes. On the canonical ticket the agent needs the order, the carrier's last scan and the seller's return window before it can decide anything, and no one of those depends on another.

Sequentially that is 0.4 seconds for get_order, 1.9 for track_parcel against the carrier's slow API, and 0.6 for search_policy: 2.9 seconds of a customer waiting. Concurrently it is 2.0, which is the slowest call plus dispatch overhead. Same tokens, same result, 0.9 seconds returned to the customer on every multi-lookup ticket. This is the first latency fix to reach for and Chapter 13 reaches for it again.

READ_ONLY = {"get_order", "search_orders", "track_parcel", "search_policy"}

def run_calls(calls):
    if not all(c.name in READ_ONLY for c in calls):
        return [dispatch(c) for c in calls]      # any write: serial
    with ThreadPoolExecutor(max_workers=4) as pool:
        futures = [pool.submit(dispatch, c) for c in calls]
    return [f.result() for f in futures]      # request order, not finish order

The two rules are in four lines. Anything that writes runs serially, because a refund and a return booked at the same instant cannot be reasoned about afterwards. And results are appended in the order the model requested them, never in the order they finished, because the transcript is what the model reads next — a scrambled history attaches the carrier's answer to the policy question and the run reasons over a fiction.

None of that helps until the model asks for the calls in one turn, and left to itself it often asks for them one at a time. Two changes fixed that at Sundry, and both belong to Chapter 3 rather than to this chapter: a line in the system prompt saying independent lookups should be requested together, and tool descriptions that state which tools do not depend on each other. The share of multi-lookup tickets issuing a batched turn went from 34% to 81%, and the 0.9 seconds only exists on those.

Parallel Agents Are Not

A second agent running at the same time is a second everything: its own system prompt and tool schemas on every one of its turns, its own transcript growing turn by turn, its own budget. Nothing overlaps except the wall clock. Sundry measured a three-branch fan-out over three sellers on a bulk claim and got a 40% improvement in wall-clock time for 3.2 times the tokens.

Both numbers are worth staring at. The wall clock did not divide by three, because the branches were uneven and the run waits for the slowest one, which is a fact of every fan-out and not a defect of this one. And 3.2× is not a rounding error on a cost report — it is the difference between a ticket the business can afford at 4,200 a week and one it cannot. Parallel agents buy latency with money, at a rate you have to measure before you can defend.

Coordination Failure

Two branches reaching contradictory conclusions is an inefficiency. Two branches acting on the same order is a data-integrity problem, and it arrives without an error message. On one bulk claim a branch assessing the damaged item started a return while a second branch, working the same order from the billing angle, offered a replacement — so the buyer got a collection booking and a replacement dispatch for a unit they had already said they did not want, and both branches recorded success.

The rule that prevents it is single-owner writes regardless of how many branches read. Branches gather and recommend; one component executes. That is the same rule as the parent-owns-the-money rule two topics ago, and it holds for the same reason: an approval ceiling enforced in three places is three ceilings, and three ceilings under $150 is a $450 ticket.

When Parallel Agents Pay

Genuinely independent work with a real deadline. When a seller's warehouse flooded, 260 open orders needed an individual assessment before the next dispatch window closed at 18:00 — every one a separate order, a separate buyer and a separate decision, with nothing flowing between them. Eight at a time finished in 24 minutes where one at a time would have taken over three hours, and the 3.2× token multiple was the correct price for making the window.

Compare that with a single ticket, which reads sequentially because it is sequential: you cannot decide the refund before you know the policy, and you cannot pick the policy before you know whether the seller is a marketplace seller. Fanning out over a dependency chain buys nothing — the second branch waits for the first anyway, and you have paid for both. Ask for the deadline before agreeing to any agent-level fan-out. If nobody can name one, the work is not urgent enough to be worth 3.2×.

Fan-Out Control

Three controls, all of them boring, and all of them missing from the first version of anything. A concurrency cap, because 260 branches launched at once hit the provider's rate limit and everything queues behind the retries — a self-inflicted outage that presents as slowness. A per-branch budget, because 260 branches at 3 turns and 4,000 tokens is a number you can price in advance, and 260 branches at the ticket allowance is not. And explicit failure semantics: when one branch fails and four succeed, the aggregate carries a per-branch status list, so a partial result is reported as partial rather than as four successes and a silence.

Sundry's flood job ran at 8 concurrent branches, 3 turns each, with failures collected rather than raised. Nine of the 260 came back aborted — three carrier timeouts, six orders whose seller record had gone missing — and those nine went to a person with their reasons attached. The alternative, which is what the first version did, was a run that reported 251 assessments and no mention of the other nine at all.

Aggregation is where a fan-out keeps its value or throws it away. Concatenating ten branch findings into the parent's context is ten thousand tokens of precisely the material Topic 54 said to keep out, so every branch returns the same structured summary a subagent does — status, coverage, finding, confidence — and the parent reads a table rather than ten essays. That also makes the partial-failure case ordinary: nine rows marked aborted are visible at a glance in a way nine missing paragraphs never are.