Three Things People Call Memory
"Memory" is one word for three engineering problems that share nothing else. What was said in this conversation. What this task has done so far. What the system knows about a customer across months. They differ in lifetime, in schema, in who writes them, in who is allowed to read them, and in what a wrong entry costs — and merging all three behind a single memory object is how you get an agent that has lost the refund it issued eleven minutes ago while confidently recalling a preference the buyer never expressed.
The model holds none of them. It is the pure function from Chapter 1: messages in, one message out, nothing retained on the other side. Every kind of memory in this chapter is therefore storage you own, plus code that decides what goes back into the context on the next turn. That reframing turns a vague question — how does the agent remember? — into four you already know how to answer: what shape is it, which store holds it, how long does it live, and who may read it.
Conversation State
The first kind is the transcript of this thread: the messages as sent, the tool requests the model made, and the results your dispatcher fed back. You own it, you re-send it in full every turn, and it is the only one of the three the model ever sees whole. A Sundry ticket that runs to the twelve-turn limit peaks at 13,380 tokens of context and bills close to 88,000 input tokens across the run, which is why Chapter 5 treats this store as a budget rather than as a log.
What it is not is a record of what happened. Compaction rewrites it. A customer who replies on Thursday resumes against a summary generated on Tuesday, and a summarizer optimizing for length is exactly the component most likely to drop the clause about the $40 already refunded. Conversation state is the agent's account of the ticket, and accounts get shortened. Anything you would have to defend in a reconciliation meeting cannot live only here. Topic 31 takes this store on its own terms: what to write down, keyed by what, and for how long.
Task State
The second kind is the structured facts of the work in progress: which ticket, which order, which items are in scope, what has been decided, and — the part everything turns on — what has actually been done. Sundry's row for the canonical ticket carries the order SU-88421, the seller Ashcombe Furniture, one oak shelving unit at $118.00, the decision to refund rather than replace, and one entry per completed side effect with the idempotency key that produced it.
This belongs in a table with a schema, never only in prose, because irreversible actions live here. The test takes ten seconds. After issue_refund returns, what in your system knows that a refund happened? If the honest answer is "the tool result message in the transcript", Chapter 3's double refund has been re-armed by a different route: one compaction pass, one restart, one thread resumed from a summary, and the record of $118 leaving a seller's balance is a clause somebody's summarizer decided was not important.
Code writes task state, after each write tool returns, from the tool's own response. The model is never asked to maintain it. Ask a model what it has done and it answers from what the context suggests — a fluent reconstruction that is usually right and is not an audit trail, which is a distinction Chapter 13 makes with traces.
Long-Term Memory
The third kind outlives the ticket: this buyer chose a replacement over a refund twice, this seller disputes damage claims, this delivery address has failed twice with the same carrier. It makes a demo feel remarkable. It is also the only one of the three where being wrong follows a named person around for months, and the only one with a legal surface — a stored characterization of a customer is personal data, has to be explicable to the person it describes, and has to be reachable by a deletion request.
Topic 35 covers it in full, including the two incidents Sundry had in the first month and why the feature shipped last and narrowest of anything in this chapter. What matters here is only that it shares no store, no retention rule and no access policy with the other two kinds, and that treating it as "the same memory, just for longer" is how the retention question stops being asked.
Retrieved Knowledge Is Not Memory
The policy library is not something the agent remembers. It is something it looks up. That distinction sounds pedantic right up until somebody in a planning meeting proposes fine-tuning the model on the policy documents so it "just knows the rules" — a proposal that sounds like memory and is in fact a very slow, very expensive index with no provenance.
Count what that trade loses. A supplement edited on Tuesday stays wrong until the next training run, on a library that changes weekly. There is no document title or effective date attached to the answer, so Chapter 9 cannot check the citation automatically and a human reviewing an escalation cannot see where the rule came from. And there is no scope filter, so Sundry's own 30-day window and a seller's statutory 14-day supplement blur into one averaged notion of a return window — which is the drift wound rebuilt from scratch, at greater cost, with the evidence removed. Keep the two words apart in code and in conversation, and the proposal does not come back a second time.
Choosing Storage Per Kind
Four problems, four stores, because the questions asked of them differ. The transcript is appended to and read whole. Task state is read and updated by key. Long-term facts are queried by subject with a date filter and need a review path. The policy index is searched by meaning and rebuilt when documents change.
| Kind | What it holds | Store at Sundry | Written by |
|---|---|---|---|
| Conversation state | Messages, tool requests, tool results | Postgres rows, hot copy in Redis | The loop, every turn |
| Task state | Order, items, decisions, completed actions | Postgres table with a schema | Code, after each write tool |
| Long-term facts | Dated observations about buyers and sellers | Separate table, with expiry and review | An extraction step, reviewed |
| Retrieved knowledge | Policy documents and their clauses | Search index, rebuilt on change | The reindex job |
Read the last column rather than the first. Four different writers means four different ways to be wrong, and putting them in one store does not merge the failure modes — it only makes them harder to tell apart at three in the morning. Access is the strongest argument of all: an engineer debugging a stuck ticket needs the transcript and the task state, and has no business at all reading a store of dated observations about a named customer.
What Sundry Stores
Sundry's version is four stores and one interesting detail — the list of completed actions inside task state, which the rest of the system treats as the record of what happened. Here is the row for the canonical ticket, after the refund that timed out and was retried.
{"ticket": "T-40219",
"order": "SU-88421",
"seller": "Ashcombe Furniture",
"seller_type": "marketplace",
"decision": "refund", # the model's call, recorded once
"actions_taken": [ # appended by code, after each tool returns
{"tool": "issue_refund", "key": "sha256:9f2c…", "amount_cents": 11800,
"outcome": "performed", "refund_id": "RF-77120", "at": "14:32:07"},
{"tool": "issue_refund", "key": "sha256:9f2c…", "amount_cents": 11800,
"outcome": "already_performed", "refund_id": "RF-77120", "at": "14:32:19"}]}
Read the two action entries at the bottom. They carry the same idempotency key, because Chapter 3 derives it from the ticket, the order, the item and the amount rather than generating it at call time. The first entry moved $118.00. The second, twelve seconds later after a timeout and a retry, matched the existing key and returned already_performed without moving anything. Neither entry was written by the model; the dispatcher wrote both when the tool returned.
That is the entire argument for separating task state from conversation state. Compaction can rewrite the transcript of this ticket into three sentences and nothing above changes. When finance reconciles seller balances on Friday, they read rows, not recollections — and the question "has this refund already gone out" is answered by a lookup on a key rather than by asking a language model what it remembers doing.
- Keeping task state only in the transcript — compaction or a resumed thread drops the line recording an action that moved money, and the loop then has no way to know the refund already went out.
- Treating documents as memory to be loaded — the whole policy library in context is hundreds of thousands of tokens billed on every turn, stale the week a seller edits a supplement, and precisely how Chapter 5's drift was manufactured.
- Using one store for all three kinds — retention windows, review steps and access control differ sharply, and the kind that most needs review, dated observations about named people, inherits the rules written for a transcript nobody audits.
- Trusting the model to track what it has done — asked whether it refunded the buyer, it answers from what the context suggests, which is a fluent reconstruction rather than an audit trail (Chapter 13).
- Keep an explicit task-state object with a schema, and update it in code after every write tool returns, from the tool's response rather than from the model's account of the turn.
- Store the transcript as application data with an owner, a schema and a retention window, on the same terms as any other customer record.
- Separate retrieval from memory in the code and in the vocabulary, so that "let us fine-tune on the policy documents" does not survive its first meeting.
- Make the decision record the source of truth for what happened, and point every reconciliation, eval and support tool at it rather than at the prose.
Knowledge Check
The agent has just issued a $118 refund. Where does the record of that action belong?
- In the conversation transcript, since the tool result already states what the refund tool returned
- In task state, as a structured action entry written by code once the tool returned its result
- In long-term memory, so that future tickets from the same buyer can see the refund history
- In the retrieval index, alongside the policy passages the agent used to reach the decision
Why must task state have a schema rather than living as prose the model maintains?
- Irreversible actions live in it, and the model reports what the context suggests rather than what ran
- Structured fields cost far fewer tokens per turn than the equivalent prose description would
- A schema prevents the model from ever misreading the state, since typed fields are entirely unambiguous to it
- Model providers validate structured state on their side and reject a run whose fields conflict
Somebody proposes fine-tuning the model on Sundry's policy library so the agent "just knows the rules". What is the strongest objection?
- Training runs are expensive and slow, so the library would be costly to keep loaded that way
- The library is simply far too large for any training run to absorb without losing most of the clauses
- Baked-in text goes stale weekly, cannot be cited, and cannot be filtered to this seller's scope
- A fine-tuned model can no longer call tools, so the rest of the agent's surface stops working
Compaction rewrites a 40-turn thread into three sentences. What still knows the $118 refund went out?
- The summary itself, since a compaction step preserves every action the agent has taken
- The model, which retains its own earlier decisions across all the turns of a single ticket
- The retrieval index, which recorded the passages that justified this refund decision at the time
- The task-state row, whose action entry and idempotency key compaction never touches
You got correct