Chapter Six · Memory and Retrieval

Memory and Retrieval

The policy library is five Sundry-wide documents plus roughly 2,000 per-seller supplements, it changes every week, and 99% of it is irrelevant to any one ticket. Six topics on giving the agent a way to look things up instead of being told everything: the three unrelated problems the word "memory" covers, conversation state as plain application data, retrieval as a tool the agent chooses to call, what vector search is good and bad at, the three ways retrieval fails, and an honest account of writing facts back about customers.

6 topics

Chapter 5 named the drift, measured it and slowed it down. It could not cure it, because the cure was never in the prompt. The agent kept quoting Sundry's own 30-day return window at buyers whose seller runs the statutory 14 days for one flat reason: both documents were in the context, and nothing in the buffer said which one governed this seller. Budgeting, compaction and careful ordering can decide what the model sees; none of them can add the fact that decides between two things it already sees.

This chapter fixes it at the source. search_policy becomes a tool the agent calls after it has read the ticket, passages come back carrying their document title, effective date and scope, and every policy-based answer has to name the clause it used. On the 32 tickets in the 120-ticket eval set whose answer turns on a policy document, policy-correct resolutions went from 20 to 26 — 63% to 81%, six tickets, and the second wound closed. Chapter 9 is where it is proved closed and kept closed.

The other half of the chapter is the word "memory" itself, which covers three unrelated problems: what was said in this thread, what this task has done so far, and what the system knows about a customer across months. Three problems, three stores, three retention rules. The most useful page here is the one about failure — retrieval misses in three distinct ways, and at Sundry the most common one was the model having the right passage in context and answering from a different one, which no retrieval metric can see. The last page is the awkward one: writing facts back about customers moved resolution by two points and produced two incidents in its first month.

Three problems called memory, and a fourth thing that is not memory at all
Conversation statewhat was said in this thread
The messages as sent, re-sent whole on every turn and rewritten whenever the thread outgrows its budget. It is the agent's account of the ticket, and accounts get shortened.
Task statewhat this task has done
Order, decision, and one row per completed side effect, written by code the moment a tool returns. An irreversible action lives here or it does not survive a summary.
Long-term factswhat is known across months
Dated observations about a buyer or a seller, each with the ticket it came from. Two points of resolution and two incidents in its first month, which is why it ships last and narrowest.
Retrieved knowledgenot memory at all
The policy library is looked up, not remembered. Passages arrive carrying a document title, an effective date and a scope — the fields that took the 32 policy tickets from 20 correct to 26.

Topics in This Chapter

Topic 30
Three Things People Call Memory
Conversation state, task state and long-term facts differ in lifetime, schema, owner and blast radius, and merging them into one API is how an agent forgets the refund it issued eleven minutes ago. Plus the fourth thing that is not memory at all: the policy library, which is looked up rather than remembered.
Memory Kinds
Topic 31
Conversation State
A support ticket lasts three days and four messages, so the transcript has to survive deploys, restarts and a customer who replies on Thursday. What a run record must contain to be replayable, why the prompt and model versions belong in it, and the per-ticket lock that stops two runs issuing the same refund.
Persistence
Topic 32
Retrieval as a Tool
Prefetching runs one search before the model has read anything and pays for it on every ticket; a tool lets the agent ask a specific question once it knows the case, ask again, or skip searching entirely. Provenance on every passage, a citation requirement, and the 63% to 81% move that closed the drift.
Retrieval
Topic 33
Embeddings and Vector Search
Why "it arrived smashed" finds a document that says "damage in transit", and why the same mechanism is worse than a keyword index at finding order SU-88421. Hybrid search as the configuration that survives production, and the two operational facts that turn an embedding model change into a migration.
Vector Search
Topic 34
Why Retrieval Fails
Never retrieved, retrieved but outranked, retrieved and then ignored — three failure kinds, three different fixes, and ten minutes of trace reading to tell them apart. Includes the recall trap, where raising the passage count improves every retrieval metric and lowers the number of tickets answered correctly.
Failure Modes
Topic 35
Long-Term Memory
"Chose a replacement twice" is a fact and "prefers replacements" is a guess, and the distance between those two sentences is the entire safety margin of the feature. Write discipline, compounding error, the never-store list written before launch, and the two-point result set against what it cost.
Long-Term Memory