# What is context rot? Why AI models lose accuracy as context grows

> Studies from Chroma, the Lost in the Middle paper and two benchmarks show accuracy falling as inputs grow, and Anthropic and Perplexity describe how agents work around it.

URL: https://claudegpt.si/news/what-is-context-rot  
Published: 2026-11-19T10:57:32.706Z  
Author: ClaudeGPT Si Newsroom  
Section: Research  
Topics: Context rot, Context window, AI agents, Anthropic, Perplexity, Chroma  
Publisher: ClaudeGPT Si

## Key takeaways

- Context rot is the drop in a model's accuracy as its context grows, and it starts well before the context window is full.
- Chroma tested 18 models in July 2025 and found performance fell with input length in every experiment.
- Teams fight it by keeping context short: compaction, memory files, sub-agents with clean contexts, and retrieval on demand.

Context rot is the drop in an AI model's accuracy as the text in its context grows, even when everything it needs is still there. It shows up long before the [context window](https://claudegpt.si/news/what-is-a-context-window) is full, and it hits AI agents hardest, because a long task keeps piling search results, files, notes and corrections on top of the original instructions.

The term took off with a July 14, 2025 [technical report from Chroma](https://www.trychroma.com/research/context-rot), the company behind the open-source vector database. On November 18, 2026, Perplexity published its own guide, ["Context rot in AI agents"](https://www.perplexity.ai/hub/blog/context-rot), which describes it as accuracy that slips away as an agent's context fills up. As Perplexity puts it: "Context rot doesn't trigger an error message."

## What context rot means

A model's context is everything it can see in one request: your prompt, the conversation so far, tool results and documents. Vendors now sell windows of about a million tokens, so it is tempting to assume a model reads the last token as carefully as the first. It doesn't.

Chroma's report states the problem directly: models "do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows." Perplexity stresses that the decline starts while everything still fits. It also separates context rot from context poisoning, where wrong information gets into the context. An agent with only accurate information can still rot.

[Anthropic's engineering guide to context engineering](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents), published September 29, 2025, explains part of the why. In a transformer, every token attends to every other token, so n tokens make n² pairwise relationships. Anthropic calls the result an "attention budget": the more text there is, the thinner the model's attention spreads over each piece.

## The evidence from Chroma, Lost in the Middle and two benchmarks

Several independent studies point the same way.

**Chroma's context rot report.** Kelly Hong, Anton Troynikov and Jeff Huber tested 18 models: five from Anthropic (including Claude Opus 4 and Claude Sonnet 4), seven from OpenAI (including o3 and GPT-4.1), three from Google (including Gemini 2.5 Pro) and three of Alibaba's Qwen3 models. Performance fell with input length in every experiment. Some findings:

- When the question and the hidden answer shared fewer words and ideas, accuracy fell faster as the input grew.
- Adding one distractor, a passage on the same topic that doesn't answer the question, lowered accuracy. Four distractors lowered it more.
- On LongMemEval, a test of question answering over chat history, every model did much better with a focused prompt of about 300 tokens than with the full version of about 113,000 tokens.
- Asked to copy a list of repeated words with one word changed, every model got worse as the list grew from 25 to 10,000 words. Some stopped early, some ran on to their output limit, and some wrote words that weren't in the input.
- Oddly, shuffling the sentences of the filler text improved results for all 18 models. Text with a logical flow made the answer harder to find.

Chroma notes the limits: the tasks were deliberately simple, the mechanism is unexplained, and real agent work is likely harder.

**Lost in the middle.** A 2023 paper, ["Lost in the Middle"](https://arxiv.org/abs/2307.03172) by Nelson F. Liu, Percy Liang and colleagues, gave models 10, 20 or 30 documents with one containing the answer. Accuracy was highest when the answer sat at the start or end and dropped when it was in the middle. With 20 documents, GPT-3.5-Turbo scored 53.8% with the answer in the middle, below the 56.1% it scored with no documents at all.

**NoLiMa.** The [NoLiMa benchmark](https://arxiv.org/abs/2502.05167), accepted at ICML 2025, removes the literal word overlap between question and answer that makes simple retrieval tests easy. Of 13 models that claim at least 128,000 tokens of context, 11 fell below half of their short-context score at 32,000 tokens. GPT-4o, one of the best, dropped from 99.3% to 69.7%.

**RULER.** The [RULER benchmark](https://arxiv.org/abs/2404.06654) tested 17 models that claim 32,000 tokens or more across 13 tasks. Only half held up at 32,000 tokens.

The numbers in these studies come from models that are now one to three years old. Newer models may do better, but none of the sources here shows the problem gone.

## How context rot shows up in AI agents

Perplexity lists four signs in long agent runs:

1. The agent loses track of the original instructions, such as a budget or an output format.
2. It misses details from the middle of the task.
3. It confuses similar items or follows distractors that look relevant.
4. It makes errors that are hard to spot: confident wrong answers, or claims that information isn't there when it is.

The last one is the dangerous part. A long run that produces a clean-looking report gives no signal that a constraint from step one was dropped. And, as Perplexity notes, every correction you type adds more text to a context that is already too long.

## How teams fight context rot

The fixes share one idea: keep the model's working context short and relevant, and keep everything else somewhere it can be fetched.

**Compaction.** When the context nears its limit, summarize it and start a fresh window with the summary. Anthropic describes this in its context engineering guide. Clearing old tool results is a light version. Anthropic's [context editing feature](https://claude.com/blog/context-management), launched in beta on September 29, 2025 with Claude Sonnet 4.5, does this automatically. Anthropic says that in a 100-turn web search test it let agents finish runs that otherwise failed and cut token use by 84%.

**Memory outside the window.** The agent writes notes, to-do lists and progress to files and reads them back only when needed. Anthropic's memory tool works this way, and Anthropic reports that memory plus context editing improved an internal agentic search test by 39% over baseline. Perplexity says each of its tasks has its own workspace where the agent saves progress to files.

**Sub-agents.** A lead agent hands parts of the job to sub-agents, each working in a clean context, which return short summaries. Both Anthropic and Perplexity describe using this.

**Retrieval on demand.** Rather than loading everything up front, the agent keeps references such as file paths or search queries and pulls the data in when needed. Anthropic calls this just-in-time retrieval. Perplexity says it passes the model only the sentences of a web page that are relevant to the query.

Perplexity also gives advice for people using agents:

- Write the task as an assignment, with the requirements and output format spelled out.
- Split large projects into focused tasks and pass only finished results between them.
- Point the agent at the right files, sites and connectors, and turn off the ones it doesn't need.
- Ask for a final check against the key constraints, and review at checkpoints.
- Start a new task when one goes off course, carrying over only the good output.
- Save instructions you repeat as a reusable Skill.

## Why it matters

Bigger context windows are a selling point, and they are useful. But the research says capacity and attention are different things. A model that accepts a million tokens doesn't treat the millionth like the first. For anyone building agents, the practical rule from all of these sources is the same: put less in the window, and put the most important things where the model will see them.

## Frequently asked questions

### What causes context rot in LLMs?

Every token in a transformer attends to every other token, so attention spreads thinner as the input grows. Anthropic calls this a limited attention budget. Distractors, similar-looking information and where facts sit in the context make it worse.

### Is context rot the same as running out of context window?

No. Context rot starts while everything still fits in the window. Perplexity separates it from context poisoning too: an agent with only accurate information can still lose accuracy as the context fills.

### What is the lost in the middle problem?

A 2023 paper by Nelson F. Liu and colleagues found models answered best when the relevant document was at the start or end of the input and worst when it was in the middle. With 20 documents, GPT-3.5-Turbo did worse with the answer in the middle than with no documents.

### How do you prevent context rot in AI agents?

Keep the working context short: summarize old context (compaction), save notes to files, hand subtasks to sub-agents with clean contexts, and load data only when needed. For users, Perplexity advises focused tasks, clear requirements and checkpoints.

## Sources

- [Context rot in AI agents: why long tasks lose accuracy and how to prevent it](https://www.perplexity.ai/hub/blog/context-rot), Perplexity
- [Context Rot: How Increasing Input Tokens Impacts LLM Performance](https://www.trychroma.com/research/context-rot), Chroma
- [Effective context engineering for AI agents](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents), Anthropic
- [Managing context on the Claude Developer Platform](https://claude.com/blog/context-management), Anthropic
- [Lost in the Middle: How Language Models Use Long Contexts](https://arxiv.org/abs/2307.03172), arXiv
- [NoLiMa: Long-Context Evaluation Beyond Literal Matching](https://arxiv.org/abs/2502.05167), arXiv
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://arxiv.org/abs/2404.06654), arXiv
