Who Plans
Between a fixed pipeline and a free-running loop there is a spectrum, and almost everything that works in production sits somewhere along it rather than at either end. The useful question about a piece of work is not whether it should be agentic. It is which parts of the sequence are known before the request arrives, and which can only be decided after something has been read.
Sundry's queue is not one kind of work. Order status is four thousand near-identical questions a week; a damage dispute against a marketplace seller is a small investigation. Running both through the same control structure means one of them is paying for flexibility it never uses, and the whole of this page is about noticing which one.
The Spectrum
Four points on it are worth naming, and the only fair way to compare them is to run one ticket through all four. Order SU-88421 is the usual one: a four-shelf oak unit at $118.00 from Ashcombe Furniture, delivered 10 March with a cracked side panel and charged twice for delivery. Three intents — money back on the delivery charge, something done about the damage, and no replacement wanted.
| Control style | Model calls | Wall clock | Cost | How it fails on this ticket |
|---|---|---|---|---|
| Fixed pipeline | 1 | 1.1 s | $0.02 | Picks one intent at the classifier and is silently wrong about the other two |
| Pipeline, model-chosen branch | 2 | 2.4 s | $0.05 | Chooses damage or billing correctly, then still answers only that one |
| Plan-then-execute | 8 | 11 s | $0.23 | Executes a plan that stopped being right when the carrier scan came back |
| Free-running loop | 9 | 8 s | $0.21 | Handles all three, and on roughly one run in eight forgets the dullest one |
Read the last column first, because it is the one that decides. The two pipeline rows are cheap and fast and structurally incapable of this ticket — no amount of prompt work fixes a design that commits to a single class before reading. The two open-ended rows both handle it, cost roughly ten times as much, and fail in different places: the plan degrades when reality moves under it, and the loop degrades when it loses track of what it was doing. Neither of those is a bug to be fixed. They are the failure profiles you are choosing between.
Code Owns What Is Known
Some sequences never vary, and every one of them belongs in ordinary code. A return at Sundry is a label, then a courier pickup, then a return record, in that order, every time, so it is one tool rather than three (Chapter 3). The refund source is not a judgement call either: money for a marketplace order comes out of the seller's balance, money for own-stock comes out of Sundry's account, and the matrix that decides is two columns wide and has no interesting cases in it.
Tidiness is not the argument for putting those in code. Code can be tested with fixtures, reviewed by a person who does not have to reason about probability, and changed with confidence on a Friday. A rule expressed in the system prompt is a strong suggestion evaluated by a component that is right most of the time; the same rule expressed as a branch in the dispatcher is a fact. When a compliance reviewer asks how Sundry guarantees that a marketplace refund never comes out of the wrong account, "we asked the model nicely" is not an answer anybody accepts twice.
The Model Owns Judgement Over Language
What the model is better at is reading. It notices that a paragraph carries three separate problems when the customer did not number them. It works out that "I just want it gone" means no replacement, which no classifier at Sundry had a label for. It recognizes that a case is unusual — the seller has closed their account, the parcel was signed for by a neighbour, the policy library has nothing on this — and that recognition is the trigger for escalate_to_human rather than an invented answer.
That is a narrow and valuable job: turn unstructured prose into a small number of structured facts, and flag the cases the structure does not fit. Everything downstream of that — validating, ordering, authorizing, recording — is work your code was always going to do better. The split is not fifty-fifty and it was never meant to be.
Plan-Then-Execute
Asking the model for the whole plan before anything runs costs one turn and buys an artefact. The artefact is inspectable: you can validate it against the tool surface, price it, count the irreversible steps in it, and show it to a human for approval instead of interrupting on every action.
{
"ticket": "T-40219",
"order": "SU-88421",
"subtasks": [
{"id": 1, "goal": "refund the duplicate delivery charge",
"tools": ["get_order", "issue_refund"], "reversible": false},
{"id": 2, "goal": "settle the cracked panel under the seller's policy",
"tools": ["search_policy", "start_return"], "reversible": false},
{"id": 3, "goal": "confirm no replacement is being sent",
"tools": [], "reversible": true}
]
}
Three subtasks, each naming the tools it expects to need and whether it changes anything the customer would notice. That structure is what makes validation possible before execution: Sundry's checker confirms every named tool exists in the surface, refuses any plan whose combined refund exposure crosses the $150 ceiling, and requires that every irreversible subtask cite a policy document before it is allowed to run. A plan that fails those checks never executes — the run goes back for a second plan, or straight to a human if the failure was the ceiling.
The cost is one extra model call and one real weakness: plans stop matching reality. Subtask 2 assumed a return was possible, and on this ticket the carrier scan came back showing the parcel was never marked delivered, which changes the whole case. A plan is a snapshot of what the model believed at turn one, and a system that executes it blindly is worse than one that improvised. Detecting that divergence is the next topic's subject, and it is the price of the artefact.
Routing by Task Type
Sundry does not pick one style. The classifier that already splits the queue picks a control structure with it, and each row of that table was justified with numbers from the eval set rather than from a design meeting.
| Task type | Share of queue | Control style | Model calls | Cost |
|---|---|---|---|---|
| Order status and delivery | 55% | Fixed pipeline | 1 | $0.03 |
| Product questions | 12% | Pipeline, model-chosen branch | 2 | $0.07 |
| Returns and refunds | 25% | Plan-then-execute, plan validated | 7 | $0.24 |
| Everything else | 8% | Free-running loop | 9 | $0.30 |
The reasoning per row is short. Order status has a known sequence and a template, so it gets neither a loop nor a plan. Product questions need one judgement — which policy or product document answers this — and then a fixed answer path. Returns are where money and irreversible actions live, so the plan exists to be approved and checked before any of it happens. The residual 8% is genuinely unenumerable, and it gets the expensive shape because nothing cheaper works on it.
Routing alone moved the blended cost from $0.14 a ticket to $0.109 — about $0.11, and well over a third of the remaining distance to the six cents Chapter 13 finishes at. It also moved the risk: the 55% cheapest path now touches read-only tools and a template, so the tickets with money attached are the only ones running under a structure that can spend it. That second effect is worth more than the money and shows up in no cost report.
Plan-then-execute — one turn produces a plan you can validate against the tool surface, price against the $150 ceiling, and put in front of a human as a single approval instead of six. It degrades when reality diverges from what the model believed at turn one, and it needs an explicit replanning step to survive that.
Improvisation — the model decides the next action from whatever the last tool returned, so it absorbs surprises without any machinery at all. It is much harder to review before the fact, because there is no artefact until the run is over and the actions have already happened.
Sundry uses a plan when the actions are irreversible, so a person can approve the plan rather than each step (Chapter 12), and improvisation when the work is read-only and a wrong turn costs a lookup.
- Letting the model decide a sequence that is legally or operationally fixed — the refund source rule is a two-column matrix, and a model that gets it wrong once has taken money from a seller who did not owe it.
- Asking for a plan and then executing it without validation — the artefact's entire value is that it can be checked before anything happens, and an unchecked plan is just a slower improvisation.
- Running a free loop over work whose steps are known — you pay nine model calls and eight seconds for a sequence a template already produced in one call, on 55% of the queue.
- Choosing one control style for the whole queue — the work is not homogeneous, so a single structure is either overpaying on the easy majority or failing on the hard tail.
- Write down, per task type, which parts of the sequence are fixed and which are open, before choosing a structure for it.
- Put fixed sequences in the dispatcher and test them with fixtures, the way you would test any other branch in your service.
- Validate every plan against the tool surface, the policy library and the $150 ceiling before executing a single step of it.
- Revisit the routing table against eval numbers on a schedule, and move a class between rows when the numbers say so rather than when someone argues well.
Knowledge Check
Which part of a Sundry return belongs in code rather than in the model's judgement?
- Whether the refund comes from the seller's balance or Sundry's own account
- Whether the customer's paragraph is asking about one problem or several at once
- Whether this case is unusual enough that a person should look at it instead
- Whether the customer's phrasing means they would accept a replacement item
What does asking the model for a plan up front actually buy, given that it costs a turn?
- An artefact you can validate and approve before any irreversible action runs
- A shorter run, because the model stops exploring once the steps are written down
- A guarantee that the sequence stays correct as new tool results come back
- Lower latency to the first useful message, since the work is scheduled in advance
Sundry runs 55% of its queue through a fixed pipeline and 8% through a free loop. What is the strongest argument for that split?
- Known sequences do not need to be decided at runtime, and the loop's cost only pays off where they are unknown
- A loop answers every class of ticket more accurately, so it is reserved for the tickets that matter most to the business
- Model providers charge a lower rate for requests that come from a fixed pipeline than for multi-turn agent traffic
- A pipeline handles complex multi-intent tickets better, which is why it takes the largest share of the queue
A team asks for a plan on every ticket and executes whatever comes back. What have they built?
- Improvisation with an extra turn in front of it, and no control they did not already have
- A working approval gate, because the plan gives a human something to sign off before actions run
- A deterministic system, since the sequence is now written down before any tool is called
- A cheaper system, because planning replaces the exploratory turns a loop would otherwise spend
You got correct