Compaction and Summarization
When a thread outgrows its budget, something has to give, and the usual answer is to replace the old turns with a summary. Done well it keeps a 40-turn ticket coherent at a third of the tokens. Done badly it drops the one fact the whole ticket depended on, and the agent then contradicts itself with complete confidence, because as far as it can see the contradiction never happened.
This is the topic where Chapter 3's work can be quietly undone. Idempotency keys made a retried refund safe; they do nothing about an agent that has forgotten a refund it made twenty turns ago and issues a second one as a new, well-formed, correctly-keyed request. The mitigation is structural rather than textual, and it is the reason half this page is about not using prose at all.
What a Summary Must Preserve
Four categories are non-negotiable. Decisions already made, so the agent does not re-litigate a settled question. Actions already taken, especially the irreversible ones — money moved, a return booked, stock reserved, a message sent to a seller. Constraints the customer stated, including the ones stated once in passing: "I don't want a replacement" is a constraint, and an agent that offers a replacement on turn 30 has failed the ticket regardless of what the policy says. And open questions, so the thread does not lose the thing it was waiting for.
Everything else is negotiable. The buyer's phrasing, the order of the diagnosis, the two tool calls that returned nothing useful, the paragraph where the agent explained the damage procedure — none of that has to survive. A good compaction is aggressive about texture and absolute about the four categories, which is a policy you can write down and check rather than a matter of summarizing skill.
When to Compact
Trigger on tokens, not on turn count. Turns vary by an order of magnitude in size — a track_parcel result is twenty times a "thanks, that works" — so a rule like "compact every ten turns" fires far too late on the tickets that need it and needlessly on the ones that do not. Sundry compacts when the assembled request would exceed 80% of the ticket's budget, which on the 8,000-token class means 6,400.
Compact at a boundary, not mid-decision. The worst possible moment is between a tool call and the reasoning that consumes its result: half the working state for the current step vanishes, the model restarts that step from a summary of it, and it frequently restarts it differently. Sundry compacts only after a resolution step has completed — a refund issued, a return booked, a reply sent — which is the same place a database would commit.
Who Writes It
A separate call to a cheaper model with a fixed template beats asking the working model to summarize itself mid-task. The working model is expensive, it is in the middle of a job, and asking it to switch to summarizing puts the summary into the same context you are trying to shrink. A separate call is isolated, cheap, and independently testable. Sundry's compaction call costs roughly a twentieth of a working turn.
Validate the output against a required-field schema, exactly as you would any other model output that your code depends on (Chapter 2). The template asks for named fields — decisions, actions taken, customer constraints, open questions — and the schema requires each one to be present. A summary missing a required field is a failed call to be retried, not a slightly worse summary to be used. Without that check, the missing field is invisible: the transcript reads fine, and the only evidence is an agent that later behaves as though something never happened.
Structured State Instead of Prose
The alternative that fails less is to stop relying on prose for anything with consequences. Maintain a small explicit state object, updated by code at the moment each action succeeds, and let the conversational history be as lossy as the budget requires around it. The state is not summarized, not rewritten by a model, and not eligible for pruning.
{
"ticket": "T-40118",
"order": "SU-87310",
"seller_type": "marketplace",
"actions_taken": [
{"tool": "issue_refund", "amount_cents": 4000,
"item": "garden-table", "at": "turn 6", "reversible": False}
],
"customer_constraints": ["no replacement wanted"],
"open_questions": ["photo of the second item still outstanding"],
"refunded_cents_total": 4000
}
Read what that object is for. It records that $40 has already gone back to this buyer for the garden table, that the action cannot be undone, that the customer refused a replacement, and what the thread is still waiting on. It is about 120 tokens, it rides in every request as pinned material, and no summarizer is allowed near it. When the agent considers a refund on turn 26, the amount already refunded is a field it can read rather than a sentence it might have lost.
This also makes a trace readable. A prose summary tells a reviewer what a model thought happened; a state object tells them what the code recorded happening, with the tool name and the amount. During an incident those are very different artefacts, and only one of them can be diffed against the payments ledger (Chapter 13).
The Failure to Watch For
Ticket T-40118 ran long. The buyer's garden table arrived with a scratched leg, they chose to keep it, and at turn 6 the agent issued a $40 partial refund. The thread continued for another sixteen turns about a second item from the same seller. At turn 22 the request crossed the compaction threshold, and the summarizer replaced turns 1 to 14 with three sentences, one of which read "the buyer reported cosmetic damage to the table and declined a replacement". Accurate, fluent, and missing the only fact with money attached.
At turn 26 the buyer came back to the scratched leg. The agent read a context in which damage had been reported and nothing had been done about it, and did the reasonable thing: it issued $40. That call carried a fresh idempotency key, and a valid one. It was a new decision made from an incomplete record rather than a retry of anything, and every control in Chapter 3 passed it. The mitigation is the one in the previous section and there is no textual substitute for it: irreversible actions live in structured state, never only in prose.
Cost and Latency
Compaction is an extra model call, and it lands at exactly the moment a ticket is already long, already expensive and already slow. At Sundry it adds about 900 milliseconds and a fraction of a cent to the turn it fires on. That is a good trade against re-sending 6,400 tokens for the remaining turns of the thread — but it is a real cost that has to appear in the per-ticket numbers rather than hiding inside "the agent was slow on that one".
Log both sides of every compaction: the full pre-compaction context and the summary that replaced it. Storage is cheap and the alternative is an incident where nobody can reconstruct what the agent knew at turn 26, because the only copy of it was overwritten by the thing that lost the fact. Chapter 13 makes that a tracing requirement rather than a suggestion.
Prose summary — captures nuance and tone, and handles the material nobody anticipated a field for. It can also silently drop a critical fact while reading perfectly, which makes it unsafe on its own; it needs a required-field validation step before anything depends on it.
Structured state — cannot drop what its schema requires, is trivially inspectable in a trace, and can be diffed against the systems it claims to describe. It captures nothing the schema did not anticipate, so it is useless for the texture of a conversation.
Sundry runs both, split on consequence: structured state for orders touched, money moved and promises made, prose for everything else. The rule is not "prefer structure" — it is that anything with a consequence must not depend on a model having chosen to mention it.
- Compacting mid-decision — the reasoning about the current step is replaced by a summary of itself, and the model restarts that step, often reaching a different conclusion than it was two seconds from reaching.
- Letting the summary be the only record of a completed refund — that is the double-refund wound re-opened by a route idempotency keys cannot see, because the second call is a new decision rather than a retry.
- Summarizing with the same expensive model mid-task — the cost of long tickets roughly doubles, the summary lands in the context you were trying to shrink, and quality does not improve.
- Running compaction with no validation on the output — a missing required field is invisible in a transcript, and the only symptom is an agent that later behaves as if something never happened.
- Keep irreversible actions and hard constraints in structured state that compaction is not allowed to touch.
- Trigger on a token threshold at a step boundary, with a fixed template and a required-field schema on the result.
- Use a cheaper model for compaction and measure on the eval set whether it changed outcomes, not just tokens.
- Log the pre-compaction context alongside the summary so a trace can be replayed after an incident (Chapter 13).
Knowledge Check
A summarizer replaced turns 1 to 14 with an accurate three-sentence summary that omitted a $40 refund already issued. Why did idempotency keys not prevent the second refund?
- The second call was a new decision with a new key, not a retry of the first one
- The idempotency key was invalidated when the summarizer rewrote the earlier turns of the thread
- Partial refunds are issued through a separate path that does not carry an idempotency key at all
- The key had expired by turn 26, so the refund service treated the request as a first attempt
Which four things must survive compaction on a Sundry ticket?
- Decisions made, actions already taken, constraints the customer stated, and open questions
- The buyer's original wording, the tone of the thread, and the order the problems were raised in
- Every tool result in full, since any of them may turn out to matter on a later turn of the ticket
- The model's reasoning for each choice, so that a later turn can reconsider a decision on its merits
Why does Sundry compact on a token threshold after a resolution step rather than every ten turns?
- Turns vary hugely in size, and compacting mid-decision throws away the state of the step in progress
- Counting turns requires state the loop does not otherwise keep, while token counts are already available
- Providers rate-limit frequent summarization calls, so a token threshold spreads them out across the thread
- A fixed turn count breaks prompt caching, while a token threshold leaves the cached prefix intact
What does compaction actually cost, and where should that cost appear?
- An extra model call, roughly 900 ms and a fraction of a cent per ticket
- A higher per-token rate for the remainder of the thread, since the rewritten prefix is billed as new input
- A measurable drop in resolution on every compacted ticket, which is the price of fitting the budget
- Storage for the retained pre-compaction contexts, which dominates the cost of running long threads
You got correct