Stop Reasons

The response field the loop branches on is not the text. It is the stop reason: a small machine-readable value saying why the model stopped generating. The model either finished its turn, asked to use a tool, ran into the output limit, or was stopped by a safety system, and each of those needs a different action from your code.

Reading it correctly is the difference between a loop that terminates and one that does not. Get it wrong and the failure is never a stack trace; it is a buyer receiving "Let me look that order up" as the resolution of a ticket, or a run that burns twelve turns re-asking a question that was refused the first time.

The Four Outcomes That Matter

A completed turn means the model has said what it intends to say and expects a person to respond next. That is the only outcome that means "done". A tool request means the model wants your code to execute something and hand back the result; the turn is finished, the work is not. An output limit stop means generation hit the ceiling you set and the text is cut off wherever it happened to be. A filter or refusal stop means a safety system ended the turn, and no amount of waiting will change that.

Every major API reports a completed turn, an output limit and a filter stop, under different field names and with different spellings for the values, and several report additional cases on top. The tool request is the one that is not universal: on one major API it never appears as a stop reason at all — the turn reports itself as complete and the request sits in the content parts — so the adapter for that provider has to look inside the reply, not just at the field. Normalize them at the adapter boundary into your own small set of constants, and treat the provider's exact strings as a detail that belongs on the last page of this chapter rather than scattered through the loop.

Tool Request Is the Normal Case

In a working agent, most turns end in a tool request. A Sundry ticket that takes four model calls ends three of them that way, and only the fourth completes. The loop's job on a tool request is mechanical: execute what was asked, append the result as a message, and call the model again with the longer array.

Two details trip people up. First, the model frequently narrates alongside the request — text and tool request coexist in one assistant message, as Topic 07 showed — so the presence of prose says nothing about whether the turn is finished. Second, a single turn can carry several tool requests at once. When the model asks for get_order and track_parcel together it is doing the right thing: those two lookups are independent, running them in parallel saves a whole round trip, and the p95 to first useful message improves accordingly. Your dispatcher has to handle all of them.

Truncation Is Not Completion

A response cut at the output limit looks like an answer. It has a role, it has text, it parses, and it ends with a word. It is not finished, and appending it to the array as though it were poisons everything downstream: the transcript now contains a half-sentence that the model will read as its own completed thought, and the next turn continues from the middle of it.

Truncation is a first-class case with three legitimate responses, and the right one depends on why the limit was hit. If the answer genuinely needs more room, raise the output limit and call again. If the model is being verbose by instruction, ask for less — a reply format spec is cheaper than a bigger limit. If the work does not fit one response, split it, which is the compaction question Chapter 5 handles. What you must not do is silently accept it, and what you must not do twice is retry with the identical limit and hope.

Refusals and Filters

Sometimes a safety system ends the turn: the model declines, or a filter on the provider's side stops generation. At Sundry this is rare and mostly benign — an abusive ticket, a buyer quoting something unpleasant a seller wrote, a request that reads like an attempt to extract another customer's data. It is still an outcome your loop meets in production, and it needs a defined path.

That path is: log it with the ticket id, preserve the original text, and hand it to a person through escalate_to_human. What it is never is a retry. The same request produces the same refusal, three times, at full cost, and the only thing the retry loop achieves is spending the ticket's budget on a decision that was already final. If a rephrase is warranted, it is a second attempt with different content, counted and logged as such — not the generic backoff your HTTP client already does for network errors.

Branching Correctly

Write the branch as an explicit table with a defined default, and let nothing fall through it unnamed.

The last row is the one teams skip, and it is the one that takes the queue down. A provider adds a new stop reason — a new refusal category, a new limit — and a loop that maps every unrecognized value to "error" turns a handful of unusual tickets into a wave of failed runs. Mapping the unknown to escalation instead degrades gracefully: those tickets reach a human, the log line names the new value, and someone adds a row to the table on Monday.

reason = adapter.stop_reason(reply)   # normalized to our own constants

if reason == TOOL_REQUEST:
    for call in reply.tool_calls:            # every one, not just the first
        messages.append(run(call))
elif reason == COMPLETED:
    return reply.text
elif reason == OUTPUT_LIMIT:
    return handle_truncation(reply)         # never append a half-sentence
elif reason == FILTERED:
    return escalate_to_human(ticket_id, ticket.text)
else:
    log.warning("unhandled stop reason: %s", reason)
    return escalate_to_human(ticket_id, ticket.text)

In words: ask the adapter what kind of stop this was, then act on the answer rather than on the prose. A tool request loops over every call in the turn, runs each one and appends each result before the next model call — miss the second of three and the next request is rejected, because every request the model made must be answered by a result. A completed turn returns. An output limit goes to a handler that decides whether to raise the ceiling or split the work. A filter stop escalates. Anything unrecognized logs its actual value and escalates too, so a provider's new enum member costs you one warning line rather than an incident.

What Breaks When You Branch on Text

The tempting shortcut is to skip the stop reason and inspect the reply instead: if there is text, the model must have answered. That works right up until the model narrates its plan before asking for a tool, which every capable model does routinely and increasingly does by default. The loop sees text, decides the run is over, and sends "Let me look that order up" to the buyer as the resolution. Nothing errored. Nothing was logged. Resolution rate falls by a few points and nobody can say why.

The variants are all the same mistake wearing different clothes: checking whether the reply "looks like an answer", pattern-matching for a closing sentence, testing whether the text is longer than some threshold, or looking for the absence of a question mark. Each one is a heuristic over prose standing in for a field that already carries the answer exactly. This is the same discipline as branching on an HTTP status code rather than grepping the response body, and it fails for the same reason when you ignore it.