State Machines Around a Model

Some parts of a support flow must not be improvised. Identity is checked before an account changes. Policy is evaluated before a refund. Approval happens before money moves, not after it. Putting those boundaries in an explicit state machine, and letting the model act only within a state, is how a loop becomes predictable without collapsing into a pipeline.

The machine constrains the shape of the work rather than the work itself. It says what phase the run is in, which tools exist in that phase, and what must be true before the run may advance. Everything inside a phase is still the model's to decide, so this sits in the middle of the spectrum rather than retreating from it.

States and What the Model May Do in Each

Sundry's returns flow is five states: triage, investigate, decide, act, confirm. Each one exposes a different tool list, and that alone changes behaviour before any guard runs, because a smaller list is a smaller set of things to choose wrongly.

Notice what is missing from triage. There is no refund tool within reach on the turn where the agent is still working out which order the customer means, and no return can be started before a policy has been read. The nine-tool surface from Chapter 3 is never fully exposed at once; the largest list any single state sees is six.

That narrowing is worth an accuracy point or two for no work at all. Chapter 3 measured 93% correct first calls on the nine-tool surface and 97% on a four-tool subset, and the per-state lists sit between those: across the returns flow, correct first calls came out at 96%. escalate_to_human is the one exception to the narrowing — it exists in every state that calls tools at all, because a surface with no exit forces the model to invent one.

Guarded Transitions

The transition that matters is decide to act, because that is the boundary money crosses. The guard is code. It reads the decision object produced by the decide turn and checks it against data, not against anything the model asserted about itself.

def can_act(run, decision):
    order = run.state["order"]
    return (
        decision.amount_cents <= REFUND_CEILING_CENTS        # 15000
        and decision.policy_ref in run.retrieved_this_run    # cited, not recalled
        and decision.source == refund_source_for(order)     # the matrix, in code
        and (not decision.needs_human or run.approval is not None)
    )

Four checks, each answering a question about the world rather than about the text. Is the amount inside the $150 ceiling. Was the policy document the decision cites actually retrieved during this run, rather than remembered from training or from another ticket. Does the refund source match what the two-column matrix says for this order's seller type. And if the decision flagged that a person is needed, is there an approval object on the run. A decision failing any of them does not advance the state; it goes back to decide once, and to a human on the second failure.

A guard that reads model prose is not a guard. "The model said it checked the policy" is a sentence, and a sentence is exactly the artefact an injected instruction or a confident hallucination can produce (Chapter 12). Every condition above is a comparison between two structured values your code holds, which is why none of them can be talked around by anything written in a ticket.

Where the Model Still Improvises

Inside a state, nothing is scripted. In investigate the model decides whether the case needs the carrier's last scan at all, whether to ask the seller directly, which query to send to search_policy, and in what order — and on the canonical ticket it does two of those and skips the third, differently on different runs. That is the flexibility a pipeline cannot have and the reason the loop is there.

The machine is a frame, not a script. It fixes five phase boundaries and the conditions for crossing them; the number of turns spent in a phase, the tools chosen inside it, and the wording of everything are all still decided at runtime by a component you do not control. If your machine also dictates which tool runs at which turn, you have written a pipeline with extra ceremony, and the model is now filling slots in it.

Deriving the Machine From Incidents

Every boundary in Sundry's machine came from a specific failure, and keeping that provenance written down is what stops the design from growing states nobody needs. The decide-to-act boundary exists because of the double refund: a retried issue_refund paid a buyer twice, and separating the decision from its execution is what made an approval and an idempotency key have somewhere to live (Chapter 3). The "cited, not recalled" check on policy_ref exists because of the drift: the agent quoted a policy it had in context from a different query, so the guard now requires the document to have been retrieved in this run (Chapter 5).

Write the incident next to the state in the code, in one line. It is the only defence against the thing that happens to every machine of this kind — a state added after each bad week, none ever removed, until the flow has eleven phases and nobody can say which failure any of them prevents. The reverse test is the useful one at review time: delete a state on paper and ask which incident comes back. If nobody can name one, the state is protecting a case that no longer exists, and the guard inside it can usually move into a neighbouring phase as a condition.

The Cost

A machine is code to maintain, and it makes the unusual ticket harder by construction. Sundry added a sixth state after one incident — an await_photo phase that held the run until the buyer sent an image of the damage — and 3% of returns tickets stopped being completable, because the buyer never sent a photo for a case that never needed one. The state was deleted four weeks later and the check moved inside investigate as a condition rather than a phase.

That is the trade: predictability for flexibility, priced per task type. Returns get the machine because money and irreversible actions are involved and the phases are stable. The residual 8% of the queue that runs as a free loop gets no machine at all, because there is no sequence of phases those tickets share. Reviewing the machine against the escalation numbers — which states runs get stuck in, which transitions fail most — is what stops the trade going bad without anybody noticing. That review is only possible because the current state is written into the trace on every turn: 62% of Sundry's stuck runs sit in investigate, which points at retrieval rather than at the machine, and a trace without a phase in it would have left that as a hunch (Chapter 13).