Knowing When It Is Done

A loop needs to end. It should end because the work is finished, and it must end even when it is not — at a turn limit, at a spend ceiling, at a repetition guard, or at an escalation. A loop with only the first condition has no termination guarantee at all, and it finds that out on a Tuesday, on one genuinely ambiguous ticket, at whatever the context window costs.

Ending is not one event with one name. Four different things can stop a run, they mean four different things about the system, and collapsing them into a single "done" flag destroys the only signal that tells you whether the next fix belongs to the product team or to you.

Four Ways to Stop

Each stop condition gets its own terminal state, its own customer outcome and its own metric. None of them shares a code path with any other, because the whole point is that they are distinguishable afterwards.

Read the second column as four names that must appear in the trace of every run, and the fourth as four numbers that must appear on the same dashboard. On a normal week at Sundry the split is 89% answered, 6% escalated, 4% exhausted and 1% stuck. Ending answered is a claim rather than a verdict — whether the answer was correct is what the eval set in Chapter 9 decides — but it is a claim the other three endings never make.

while True:
    if run.turn >= TURN_LIMIT:       return end(run, "exhausted")
    if run.spend_cents >= CAP:       return end(run, "exhausted")
    if no_progress(run):             return end(run, "stuck")

    reply = model(run.messages, tools=tools_for(run.state))
    if reply.stop_reason != "tool_use": return end(run, "answered")

    for call in reply.tool_calls:
        if call.name == "escalate_to_human": return end(run, "escalated")
        run.messages.append(dispatch(call))

Four exits, four names, one function. Every one of them goes through end, which writes the terminal state into the trace, increments the right counter, and produces a customer message appropriate to that ending — there is no path out of this loop that leaves a run without all three. The checks come before the model call rather than after it, so a run that is already over does not pay for one more turn to discover that.

Finished, Stuck, and Gave Up

Group the four states into three readings, because that is how a person interprets the dashboard. answered is finished. escalated is the agent giving up on purpose — it read the situation, decided a person was needed, and handed over with a summary. exhausted and stuck are both the loop failing: one ran out of budget, the other went round in circles until a guard noticed.

The distinction earns its keep in what a rising number means. A rising gave up rate is a product signal: the policy library is missing cases, or a seller category is producing situations the agent has no remedy for, or a new fee is generating tickets nobody wrote a rule about. Nothing is broken; the work has changed shape and somebody in the product should hear about it. A rising stuck rate is an engineering signal: a tool started timing out, a retrieval index went stale, a schema change is making the model re-ask for the same thing. Those are two different pagers, and a system with one "done" state rings neither.

Getting this wrong is silent, which is the recurring theme of this book's failure modes. A single done flag makes an exhausted run and a resolved run identical in the metrics, so a week in which timeouts doubled looks exactly like a week in which nothing happened. The cost of separating them is four string constants.

Setting the Limit From Data

Sundry's loop limit is twelve turns, and two independent measurements agree on it. The first is quality, and Chapter 5 already made it: resolution measured 88% at 5 turns of prior history, 85% at 15, 81% at 25 and 71% at 30, so the curve is flat to about 15 and falls off a cliff past 25. Twelve turns sits inside the flat region with room to spare. That is why the limit never had to move when the curve was measured — it was already below the knee.

The second argument is about usefulness rather than quality, and it comes from the turn counts of runs that actually resolved.

Ninety-four per cent of resolved runs finish within eight turns, and 99% within twelve. That table is a share of resolutions, and it does not by itself say how a run still going at turn 13 will fare; that is a second measurement, and Sundry took it by replaying the set with the limit lifted to thirty. About one run in twenty-five was still going at turn 13 — the same 4% that ends exhausted under the limit today — and of those roughly one in five ever produced a resolved answer, which is where the 1% in the last row comes from. Meanwhile the cost per turn keeps climbing, because every turn re-sends the whole transcript — the twelfth turn of a run bills more input tokens than its first four combined. Past twelve you are paying the most expensive turns in the run for a one-in-five chance, and the resolutions it buys are one per cent of the total.

Two arguments, arrived at separately, landing on the same number. The quality curve says do not go far past the knee; the distribution says there is almost nothing out there to find. Note that this limit bounds a single run — the 20-turn rule from Chapter 5 is a different control, bounding an accumulated thread across days of customer replies, and a ticket can hit that one having never come close to this one.

Progress Detection

A limit catches a run that is going nowhere slowly. A guard catches one that is going nowhere fast, and it is much cheaper to detect than it sounds. The same tool called twice with the same normalized arguments is one signal. Three turns during which nothing new entered the subtask state or the structured record is the other. Both are a hash and a counter in the loop, and at Sundry they fire on about 2% of runs, saving an average of six turns each.

What makes a repeated call a loop rather than a legitimate retry, how to normalize arguments so a whitespace difference does not defeat the check, and the rest of the taxonomy belong to Chapter 8, which owns the failure class properly. What belongs here is the control-flow decision: the guard is an exit from the loop, with the terminal state stuck, not a warning logged while the run carries on.

What Happens at the Limit

Hitting a limit is a designed outcome, not an accident, and the design has three parts. Summarize the state the run reached — subtasks resolved, tools called, anything already irreversible. Escalate with that summary attached, so the person picking it up starts from what the agent found rather than from the raw ticket. And send the customer something true.

"I've refunded the duplicate $9.95 delivery charge and I'm checking the return options for the cracked unit with the seller — a colleague will come back to you today" is honest, names what was actually done, and does not promise a resolution nobody has. The alternative, which is what a silent limit produces, is forty seconds of nothing followed by a run that vanished from the customer's point of view while having already moved money. A limit with no defined behaviour is worse than no limit, because it fails in the middle of a decision rather than at the end of one.

Partial Success

Three subtasks and two resolved is a real ending, and it needs to survive as one all the way to the metrics. The run ends answered with the open subtask still marked open; the reply names it in plain terms; and the record counts a partial resolution rather than a success. Rolling partials into the success column is how a system reports 91% resolution to a business that is receiving second tickets from a tenth of its customers — the number is not wrong so much as it is measuring the wrong thing, which Chapter 9 takes apart properly.