The Stateless Function Underneath
The model is a pure function. A list of messages goes in, one message comes out, and nothing is retained on the other side — no session, no memory of the previous turn, no awareness that a conversation is happening at all. Call it twice with the same list and you have not resumed anything; you have asked the same question twice.
Every apparent memory in every agent product you have used is somebody's code re-sending the history. That is the mechanism rather than a teaching simplification, and once it lands most of this book stops being surprising. Conversation state exists because you store it. The context budget is real because the buffer is finite. A long thread is a growing prompt rather than a relationship.
The Function Signature
The call takes a list of messages and a list of tool definitions. It returns one message, a count of the tokens that went in and came out, and a stop reason saying why generation ended. That is the whole contract. Chapter 2 opens the HTTP request and reads each field properly; what matters here is the shape.
reply = model(
messages=[...], # everything the model will see this turn
tools=[...], # what it is allowed to ask for
)
reply.content # text, and any requests to run a tool
reply.stop_reason # why it stopped -> the loop branches here
reply.usage # input and output tokens -> the bill
Three fields come back and each one drives a different part of the system. The content is what the model produced, which may be prose for the customer or a structured request to run one of your tools. The stop reason is the field the loop branches on, and Chapter 2 spends a topic on it because branching on anything else is a bug. The usage counts are the bill, and they are the only honest input to the cost model in Chapter 13.
Send the identical list twice and you can get two different replies. That is sampling, and it is worth separating from memory right now: the variation comes from how each next token is chosen, not from the service recognizing you. Chapter 2 covers what temperature actually changes and why "make it deterministic" is not on the menu.
There Is No Session
The provider holds nothing between calls. There is no conversation object accumulating on their side, no identifier that makes turn five cheaper than turn one, and no state to expire. Each request is complete on its own, which is also why you can replay a failed turn from your logs and get a genuinely comparable run.
Hosted threads and conversation abstractions in the vendor SDKs do exist, and they are storage the vendor put in front of the same stateless call. They keep your messages and re-send them on your behalf. That is a convenience with a price tag attached: the input tokens are billed again on every turn, even though you only uploaded the text once, because underneath the abstraction the whole array still travels on every request.
The practical consequence arrives on day one of writing your own loop. It forgets everything between turns, and the reflex is to hunt for the flag that turns memory on. There is no flag. The history is your application's data, with a schema, a home and a retention policy, which is the argument Chapter 6 makes in full.
Everything Is Re-Sent, Every Turn
Turn twelve sends turns one through eleven back in full. The context grows monotonically across a run — the system prompt, the tool schemas, the ticket, every tool request, every tool result — and the bill grows with it, because the model is charged for reading all of it again. Sundry's queue makes this concrete. A plain order-status ticket takes four model calls: a search for the buyer's orders, a lookup of the one they mean, a parcel scan from the carrier, and then the answer.
| Model call | What this turn adds | Context sent | Input billed so far |
|---|---|---|---|
| 1 | System prompt, tool schemas, the customer's ticket | 1,380 tokens | 1,380 |
| 2 | The request for search_orders and its result | 2,530 tokens | 3,910 |
| 3 | The request for get_order and its result | 3,750 tokens | 7,660 |
| 4 | The request for track_parcel and its result | 4,680 tokens | 12,340 |
Read the arithmetic rather than the table. The largest context this run ever sends is 4,680 tokens, and the input you are billed for is 12,340 — two and a half times what a glance at the final prompt suggests. Nothing was misconfigured and nothing was wasted. That is simply what re-sending everything costs, on a ticket nobody would call complicated.
Stretch the same shape to Sundry's loop limit of twelve turns and it gets loud. A run that reaches the limit ends on a context of about 13,000 tokens and bills close to 88,000 input tokens, because each of those twelve calls carried most of the previous ones on its back — Chapter 2 works it turn by turn. This is why Chapter 5 is an engineering discipline with a budget rather than prompt tinkering, and why prompt caching exists at all — the repeated prefix is the single largest line item in an agent's bill.
The Model Cannot Do Anything
The model emits text. Some of that text is a structured request to run one of the tools you described, carrying a name and a set of arguments, and it is still text. Nothing happens until your code parses that request, decides whether to honour it, and calls a function. The model has no network, no credentials and no reach into Sundry's order system.
That is the entire security posture of Chapter 12 in one sentence. When an incident review says the agent issued a refund, what happened is that the model produced a request and a dispatcher you wrote executed it against a credential you granted. Authorization belongs in the dispatcher, where it can refuse. A rule in the prompt is a suggestion to a component that is allowed to be wrong.
What the Model Actually Knows
Two sources, and only two: what it absorbed during training up to a cutoff date, and what is in the context window on this call. It does not know today's date. It does not know that this buyer opened a ticket last Tuesday, what Sundry's refund ceiling is, or whether a given order shipped from a Sundry warehouse or from a marketplace seller. None of that is in the weights, and none of it arrives by magic.
For Sundry that turns into a checklist rather than a philosophical point. The current date goes in context, because "within 30 days of delivery" is undecidable without it. The buyer's identity goes in, because the agent must not answer about somebody else's order. The refund ceiling goes in, because a model that has never heard of $150 will happily reason its way past it. Leave one out and the failure is not an error message — it is a fluent, confident guess, which is much harder to notice.
Consequences You Will Meet Later
Four chapters are already implied by this page. Memory is storage plus re-sending, so Chapter 6 splits the word into three different problems with three different solutions. The context budget is a real budget with a per-turn price, which is Chapter 5. A long conversation is a growing prompt with a measurable degradation curve, which is the drift that bites Sundry in Chapter 5 and gets closed in Chapter 6. And caching exists because the prefix repeats, which is the cheapest large win in the book.
Hold on to the reframe more than the details. When someone asks why the agent did something strange, the useful first question is not what the prompt says but what was actually in the context on that turn — because that context, and nothing else, is what the function was called with.
- Assuming the provider keeps conversation state because a hosted playground appears to — the first loop you write yourself then loses everything between turns, and the mechanism gets learned backwards from a bug report.
- Writing prompts that refer to "what we discussed earlier" without re-sending it — the model does not report that it has no earlier discussion, it confabulates a plausible one and answers as if that were the record.
- Budgeting cost as turns multiplied by prompt size — the real figure is the sum of a growing prefix, which on a twelve-turn ticket is several times the naive estimate and lands as a surprise on the first full month.
- Expecting a fact stated once in turn 1 to still govern behaviour at turn 30 — position and sheer volume both work against it, which is exactly what the context-rot measurements in Chapter 5 quantify.
- Believing a tool "was called by the model" — the model asked, your code called it, and putting the authorization check anywhere other than that dispatcher leaves the ceiling enforced by persuasion.
- Log the exact message array sent on every turn while developing — most confusing agent behaviour explains itself the moment you read what was actually in the context rather than what you believe you assembled.
- Treat conversation history as application state you own, with a schema, a home and a retention policy, rather than as a vendor feature you rent (Chapter 6).
- Price a feature by the growth of its context rather than by the number of model calls — one extra tool result early in a run is paid for on every turn after it.
- State the current date, the customer's identity and the account facts explicitly in context whenever they matter, and never assume the model has them.
Knowledge Check
Your loop is about to send turn 5 of a Sundry ticket. What does the provider still hold from turns 1 through 4?
- Nothing at all, since the earlier turns are in this request only because your code re-sent them
- The full conversation, keyed by the identifier that the SDK issued when this thread was opened
- A running summary of the earlier turns, which the provider consults whenever the window gets tight
- The tool definitions, which are registered once per conversation and reused on every later call
A ticket resolves in twelve turns. Why is the input bill much higher than twelve times the first turn's prompt?
- Every call re-sends the whole conversation so far, so input is a sum over a growing prefix
- Longer threads are automatically served by a larger model tier, which is priced higher
- Output tokens are priced higher than input tokens, and the later turns produce most of the output
- Tool definitions are re-registered on every call and billed separately from the message array itself
Sundry caps refunds at $150 without human approval. Where does that ceiling have to be enforced?
- In the dispatcher that receives the request and runs the function, which is the only component able to refuse
- In the system prompt, stated as a standing rule that the model is instructed to follow on every single turn
- In the tool schema, whose parameter description sets a maximum the model is expected to respect
- In the output check that rejects any reply whose text quotes an amount above the ceiling
A buyer asks whether their return window is still open. What does the model know before you put anything in the context?
- General knowledge up to its training cutoff, and nothing about this buyer, this order or today's date
- The order record, because the schema for get_order describes every field that the tool returns
- Today's date, which the provider injects into every request before the model reads it
- This buyer's previous ticket, which stays available for the lifetime of the session
You got correct