The Agent Loop

The loop is four steps: send the context, read the response, execute what it asked for, append the result. Repeat until the model answers instead of asking, or until a limit you set stops it. That is the machine, and it fits comfortably on one screen.

Every agent framework in existence is that loop plus opinions — about how state is stored, how retries work, how prompts are assembled, how a run is resumed. The opinions are often good. They are also invisible from the outside, which is why writing the loop yourself once is worth an afternoon: after this page, a framework is something you can evaluate rather than something you have to trust.

The Four Steps

Build the message list. Call the model. Branch on the stop reason. Execute any tools it requested and append their results as messages. Then go around again with a context that is one exchange longer than the last one. Nothing in that sequence is clever, and everything later in this book attaches to one of those four points.

def run(ticket, max_turns=12, budget_usd=0.50):
    messages = [system_prompt(), user(ticket.text)]   # 1. build the context
    spent = 0.0

    for turn in range(max_turns):
        reply = model(messages, tools=TOOLS)          # 2. call the model
        spent += price(reply.usage)
        messages.append(reply)

        if reply.stop_reason != "tool_use":            # 3. branch on the stop reason
            return done("answered", reply.text)
        if spent > budget_usd:
            return done("over_budget", messages)

        for call in reply.tool_calls:                   # 4. execute, then append
            result = DISPATCH[call.name](**call.args)
            messages.append(tool_result(call.id, result))
            if call.name == "escalate_to_human":
                return done("escalated", result)

    return done("turn_limit", messages)

Read it as four moves and four exits. The message list starts as a system prompt plus the customer's ticket and only ever grows. Each pass calls the model with that list and the tool definitions, adds up what the call cost, and appends the reply. If the model stopped for any reason other than wanting a tool, the run is over and the text goes to the customer. Otherwise every requested tool is looked up in a dispatch table, executed, and its output appended as a new message carrying the identifier of the request it answers.

The branch in step three is the whole of the control flow, and it reads one field. Not the presence of text, not a keyword, not a JSON blob the model was asked to produce — the stop reason the API returns alongside the message. Chapter 2 spends a topic on that field because every other way of deciding whether the model is finished breaks on a model that narrates while it works.

Termination

A run should be allowed to end in exactly four ways. The model returns a final answer. The turn limit is reached — twelve for Sundry, and the number is a product decision rather than a default. Spend crosses its ceiling for this ticket. Or a tool signals a terminal condition, which for Sundry is escalate_to_human: once the agent has handed a ticket to a person, continuing to work on it is worse than stopping.

A loop with only the first condition has no termination guarantee at all. It will still finish most of the time, because most tickets are easy, and it will run until the context window or the month's budget stops it on the first ambiguous one. The other three are three lines of code in the first version and they bound an entire class of incidents in Chapter 8.

Hitting the turn limit is a normal outcome, not an error. It needs defined behaviour: summarize what is known and answer with it, hand the ticket to a person with that summary attached, or fail loudly to a queue somebody watches. A loop that raises an exception on turn twelve has turned a slow ticket into a lost one.

The Shape of the Growing Context

What the model sees on each turn is a single interleaved transcript: the customer's message, the assistant's request for a tool, the result of that tool, another request, another result, and eventually an answer. Tool results are messages with a role, exactly like the customer's text, and they are worth more attention than they usually get.

[
  {"role": "system",    "content": "You are Sundry's support agent. Today is..."},
  {"role": "user",      "content": "Tracking says delivered but nothing arrived..."},
  {"role": "assistant", "content": [tool_use get_order {"order_id": "SU-41207"}]},
  {"role": "tool",      "content": {seller, items, charges, description, ...}},
  {"role": "assistant", "content": [tool_use track_parcel {"tracking": "NW-9930..."}]},
  {"role": "tool",      "content": {last_scan: "delivered, left in porch, Thu 11:42"}},
  {"role": "assistant", "content": [tool_use search_policy {"query": "parcel not received"}]}
]

Two things in that array matter more than their appearance suggests. Every tool result occupies the same budget as prose and is re-sent on every later turn, so the full order record that get_order returns — items, charges, and the seller's own product description — is paid for again on turns four, five and six. And each result carries the identifier of the request it answers, which is how the model knows that the carrier scan belongs to the tracking lookup rather than to the order lookup.

Where State Lives

There are two stores, and no third one. Inside a run, state is the message list: everything the agent has seen or done on this ticket is a message in that array, in order. Between runs, state is whatever you wrote to your own database — the ticket record, the refund you issued, the fact that this buyer already got a replacement in March. The loop itself is stateless code operating on those two stores.

That is a useful property rather than an accident. Because the loop holds nothing, the same function can run in a request handler, in a worker, or in a retry after a crash, and it will behave the same way given the same two inputs. It is also why "resume this run tomorrow" is a real engineering problem rather than a flag: the message list has to be somewhere durable first, which is where Chapter 11 picks it up.

What the Loop Does Not Include

Planning is not in these four steps. Neither is memory, retry logic, evaluation, permission checking or anything resembling safety. The loop will happily call the same failing tool nine times, quote a policy that does not apply to the seller who shipped the order, spend eleven turns on a question it could have answered in two, and issue a refund it should have asked about first. None of that is a defect in the four steps; those steps do exactly what they say.

Which makes the list a map of this book. Tools and their schemas are Chapter 3, and the protocol for tools somebody else owns is Chapter 4. What goes into the context and what gets cut is Chapter 5, and where longer-lived knowledge comes from is Chapter 6. Who decides the sequence is Chapter 7, the failure classes are Chapter 8, and measuring any of it is Chapter 9. Permissions and approval gates are Chapter 12. Everything is bolted onto the loop above; nothing replaces it.

Reading a Real Trace

Take a Sundry ticket off the queue: "Tracking says delivered on Thursday but there is nothing here, and I have already asked the neighbours. Order SU-41207." On turn one the context holds three things — a system prompt naming the agent's role, today's date and its refund ceiling; the schemas for the nine tools; and the customer's text. The model asks for get_order with the id the customer supplied, which is the cheapest possible way to find out whose problem this actually is.

Turn two carries that request and the order record back in. The record says the order shipped from a marketplace seller rather than a Sundry warehouse, and it carries a carrier tracking reference. The model now asks for track_parcel. Turn three brings back a last scan reading delivered, left in porch, Thursday 11:42 — which is the carrier's claim, not a fact, and the distinction is exactly the sort of thing the model handles well and a rules engine never did.

On turn four the model asks search_policy for the missing-parcel procedure and gets two documents: Sundry's own damage-and-loss procedure, and the supplement for this particular seller, which requires 48 hours from the scan before a claim can be opened. Two documents matching one query is the normal case at Sundry, not an edge case, and Chapter 6 is about what happens when the agent picks the wrong one.

Turn five is the answer: the scan is 31 hours old, so the agent tells the buyer what the carrier recorded, states the 48-hour rule in plain words, and messages the seller to open the claim the moment the window passes. Five model calls, five contexts each larger than the last, and one exit path taken out of four. Reading a run in exactly this way — what was in the context, what was asked for, what came back — is the skill that Chapter 13 turns into tooling, and it is the fastest debugging technique in the book.