Context Rot
Long contexts degrade, and the degradation is easy to mistake for the model getting worse. Instructions from early turns lose force. Two contradictory facts sit side by side and one of them wins for reasons that have nothing to do with which is correct. Irrelevant history pulls attention away from the case. Material in the middle of a long buffer gets less weight than material at either end.
This is the second of the book's three wounds, and it is the one that produces no error, no exception and no alert. The agent answers fluently, confidently, and wrongly. This page names the four ways that happens so you can recognize them in a transcript, then measures the curve so the limits you set come from a number rather than a feeling.
Four Named Failure Modes
Naming them is the point. "The model got worse on long threads" is not a diagnosis and does not suggest a fix; "the constraint from turn 2 is losing to a tool result from turn 24" tells you exactly what to change. Four modes cover almost everything you will see.
| Failure mode | What it looks like in a transcript | What it actually is |
|---|---|---|
| Instruction decay | A rule followed for fifteen turns is quietly dropped on the sixteenth, with no mention of it | The constraint is competing with far more recent, far more specific material |
| Contradiction | The agent states one thing and then its opposite, both with full confidence | Two conflicting facts are in the buffer and nothing marks either as superseded |
| Distraction | The reply answers a question from twelve turns ago instead of the one just asked | Irrelevant history is still present and still competing for attention |
| Position effects | A fact stated in the middle of a long thread is treated as though it was never there | Attention is weighted toward the start and the end of the buffer, not the middle |
They compound rather than arriving one at a time. A long thread has usually accumulated several superseded facts, stretches of irrelevant history, and a constraint now 12,000 tokens from the decision it governs — so the failure you see is all four at once, and the mitigations below are ordered by effect rather than offered as alternatives.
None of the four is a bug in the model, and none of them is fixed by a better model. They are what happens when a fixed attention budget is spread over a buffer that keeps growing — a bigger window makes the failure available at greater lengths rather than removing it.
The Drift, Diagnosed
Order SU-90462, a garden bench from a marketplace seller. On turn 3 the agent called search_policy about return windows and got the seller supplement back: this seller runs the statutory 14 days, not Sundry's own 30. The thread then went on for fourteen more turns — carrier tracking, a request for a photograph, a message to the seller, two customer replies a day apart. On turn 16 a different search_policy call, this time about damage in transit, returned Sundry's own returns page, which carries the 30-day line in its second paragraph.
On turn 19 the buyer asked how long they had to send the bench back. The agent said 30 days. Both policies were in the context. The correct one, the supplement, sat in the middle third of a 14,000-token buffer; the wrong one had arrived three turns ago and was 9,400 tokens closer to the end. Nothing in the system prompt was violated — the rule says to call search_policy and answer from what it returns, and that is precisely what happened, from a search_policy result. There is no instruction you could add that would have caught it, because the agent did not break a rule. It picked the more recent and better-positioned of two documents it had no way to rank.
Sundry sent that answer on 41 tickets over six weeks. Every one of them was honoured when buyers came back outside the seller's window, which cost about $3,100 — and the discovery came from a seller complaining about returns they had not agreed to, not from a monitor. That detail matters more than the money: nothing in the system was watching for an answer that was fluent, well-sourced and wrong.
Measuring Degradation
The only way to know where your own curve turns down is to measure it. Sundry took the same 40-ticket slice of the eval set and ran it four times, padding each run with real prior history to a fixed length before the ticket's own turns began. Same tickets, same tools, same prompt, same model — only the length of what came before changed.
| History before the ticket | Resolution | Dominant failure |
|---|---|---|
| 5 turns | 88% | None — this slice's own ceiling on a short thread |
| 15 turns | 85% | Occasional distraction on multi-intent tickets |
| 25 turns | 81% | Position effects; the knee of the curve |
| 30 turns | 71% | Contradiction and instruction decay together |
Read the shape rather than the individual levels. Those levels belong to these 40 tickets under a padded history; they are not the agent's resolution rate on the queue or on the full eval set. It is flat to about 15 turns, bends at 25, and falls off a cliff between 25 and 30 — a 10-point drop across five turns, on tickets that come out at 88% when the thread in front of them is short. Nothing in the code differs between the top row and the bottom one.
Two operational limits fall straight out of that table, and they govern different things. The twelve-turn loop limit bounds a single run of the agent — it was originally a spend control from Chapter 2, and this curve is why it never had to move: twelve turns sits inside the flat region with room to spare. But a support thread is not one run. A ticket where the buyer replies on Tuesday and again on Thursday accumulates turns across days, and nothing in the loop limit stops the thread reaching 30. That is the case the second limit is for, and it is where the drift above actually lived.
Contradiction Is Worse Than Absence
Given no policy document, the agent says it needs to check and calls the tool again. Given the wrong policy document, it answers confidently and incorrectly. An empty context produces a hesitant agent; a contradicted one produces a wrong agent that sounds exactly like a right one, and only the second kind gets sent to a customer.
That asymmetry is the argument for precision over recall in retrieval, which Chapter 6 makes properly. Returning five documents so the right one is definitely among them feels safe and is not: four now compete with the fifth, none is marked authoritative, and the model settles the tie on recency and position. Two documents matching one query is the normal case in Sundry's policy library, not an edge case.
Mitigations, in Order of Effectiveness
Remove the irrelevant content. That is first by a wide margin, because it is the only mitigation that eliminates the competition rather than trying to win it — a superseded policy that is not in the buffer cannot be selected. For the drift that means retrieval returning one passage, chosen by seller type, instead of two that disagree (Chapter 6). Second, keep the constraint governing the current decision near that decision, which usually means late in the buffer rather than in the system prompt. Third, restate it at the point of use: right before a refund call, the loop injects the refund rules as their own message.
Fourth, cap the thread and hand off to a fresh context — the subject of the next section, and the same mechanism a subagent boundary uses in Chapter 10. Fifth, and only fifth, prompt harder. Emphasis, capitals and repeated warnings buy a small improvement that shrinks as the thread grows, which is the opposite of what you need. Anything you would be tempted to shout about belongs in code (Chapter 12).
When to Stop Extending a Thread
Sundry's rule: past 20 turns of accumulated thread, do not continue. Summarize into a fresh context seeded with the structured task state from Topic 27, and carry on there. Twenty is not a taste; it is the 25-turn knee with five turns of margin, which is roughly the length of one more customer exchange. A rule derived that way survives an argument, and it moves when the curve moves rather than when someone's intuition does.
A fresh context inverts all four modes at once. The constraints are near the top of a short buffer again, no superseded material is left to compete, the irrelevant history is gone, and the structured state carries what actually mattered — what was decided, what was done, what the customer refused. The thread the customer sees is unbroken; the buffer behind it starts again, which is the only reliable cure for every failure mode on this page.
- Attributing the degradation to the model and upgrading in response — a larger window makes the failure available at greater lengths rather than removing it, and the bill goes up while resolution does not.
- Leaving superseded documents in context "for reference" — they compete with the correct one on equal terms, and the tie gets resolved on recency and position rather than on correctness.
- Restating critical constraints only at the top of the prompt — position effects work against exactly that placement once the buffer is long, which is the placement everybody reaches for first.
- Assuming a large advertised window means usable quality across all of it — Sundry's own curve turns down at 25 turns inside a window that could hold several times that.
- Measure the degradation curve on your own eval set by padding history to fixed lengths, and set the turn limit from where it bends.
- Remove superseded material rather than annotating it, because an annotation is one more thing competing for attention.
- Restate the constraint that governs the current decision immediately before the call it governs, not only in the system prompt.
- Prefer a fresh context seeded with structured state to an ever-longer thread, and cap the thread at a length the curve justifies.
Knowledge Check
On turn 19 the Sundry agent quoted a 30-day return window at a buyer whose marketplace seller runs 14 days. What actually went wrong?
- Both policies were in context, and the wrong one was more recent and closer to the end of the buffer
- Retrieval failed to find the seller supplement, so the agent answered from the only policy it had
- The agent violated the system prompt rule about never quoting a return window from memory
- Compaction dropped the supplement from the history, leaving only the company's own policy behind
Why is one wrong document in context worse than no document at all?
- With nothing, the agent checks again; with something wrong, it answers confidently and incorrectly
- The extra document consumes budget that the correct passage would otherwise have been able to use
- Models cannot process more than one policy document per turn without degrading on both of them
- The agent refuses to answer when it detects a conflict, so the ticket stalls until a human steps in
Sundry's curve reads 88% at 5 turns, 85% at 15, 81% at 25 and 71% at 30. What follows for the thread limit?
- Cap the thread around 20 turns and start a fresh context, keeping margin below the knee at 25
- Cap the thread at 30 turns, since that is the last length at which resolution was actually measured
- Cap the thread at 5 turns, because that is the only length where the ceiling was reached at all
- Adopt the limit published with the model's long-context benchmark rather than measuring your own
Where should a constraint that governs a refund decision be placed on a 25-turn thread?
- Injected immediately before the refund call, in addition to whatever the system prompt says
- At the very top of the system prompt, where the model reads it before anything else in the request
- In the middle of the thread, close to the point where the customer first raised the refund request
- Repeated once every five turns throughout the thread, so that no stretch is ever far from a copy
A team responds to degradation on 30-turn threads by moving to a model with a much larger context window. What should they expect?
- Little improvement at 30 turns, because the failure is competition inside the buffer rather than capacity
- Resolution back at the 88% ceiling, since the degradation was caused by approaching the previous limit
- An end to truncated replies, which was the underlying symptom the degradation curve was measuring
- No further need for compaction, since threads of any realistic length now fit inside one request
You got correct