Incredible
White paper

The Attention Budget:Understanding and Mitigating Context Rot in LLMs

Originally published August 2025 · Updated September 2026

On this page
  1. Abstract
  2. Who this is for
  3. Glossary
  4. 1. Executive summary
  5. 2. The problem in practice
  6. 3. Foundations: what goes into context
  7. 4. Defining context rot
  8. 5. Evidence base
  9. 6. Mechanisms of context rot
  10. 7. Implications by use case
  11. 8. Mitigation framework
  12. 9. Measurement and governance
  13. 10. Recommendations
  14. 11. Open questions and future work
  15. References

Abstract

Large language models accept ever larger context windows. Capacity to submit tokens is not capacity to use them well. Context rot is the measurable decline in output quality as input length grows, even when the task stays fixed and the needed information remains present.

This white paper treats the context window as an attention budget and gives builders a playbook: prevent long, noisy prompts before you remediate; design products so the default path stays short; measure usable length as a portfolio of bands by task type, not one marketed maximum. Evidence on position, length with retrieval intact, agent early-exit, and usable-versus-marketed windows all press the same point. Govern the budget; do not confuse a larger maximum context window with reliable memory.

Who this is for

This paper gives product and engineering teams a shared frame and playbook for why long context fails in practice and what to change in shipping systems.

Primary readers: product managers, applied engineers, and platform teams shipping chat or agent features.

Secondary readers: technical executives and investors who need a shared vocabulary for why “we have a million-token model” does not settle reliability debates.

How to read

If you need… Start here
The short version §1, then §10
Definitions and citations §§4–5, glossary, References
Mechanisms of context rot §6
Where it shows up in products §§2, 7
What to build and measure §§8–10
Open research §11

Glossary

Token
The atomic unit of text an LLM reads and writes (roughly a word piece). Context limits and pricing are usually measured in tokens.
Context window
The maximum number of tokens a model can accept in one inference call (system prompt, user messages, tool schemas, retrieved documents, history, and tool results combined).
MCW (Maximum Context Window)
The marketed / architectural upper bound on input length (e.g., 128K, 1M tokens).
MECW (Maximum Effective Context Window)
The longest span of input, for a given problem type, before additional tokens measurably degrade output quality. MECW is task-dependent and typically far below MCW. Treat it as a portfolio of bands across task families, not one global number.
NIAH (Needle in a Haystack)
A long-context test that hides a fact (“needle”) in a long distractor document (“haystack”) and asks the model to retrieve it. Strong NIAH scores do not imply strong long-context reasoning.
Lost in the Middle (LiM)
A positional pattern where models use information at the beginning and end of the input better than information in the middle.
Context rot
Measurable decline in LLM performance as input length grows, even when task difficulty is held constant and relevant information is present.
Attention budget
This paper’s name for a practical consequence of transformer attention: as more tokens enter the window, effective use of any one token tends to thin out.
Compaction
Rewriting or summarizing accumulated context into a shorter form so work can continue in a smaller window.
Trimming
Dropping tokens (e.g., old tool outputs) rather than rewriting them; often leaving pointers or IDs behind.
Isolation
Keeping bulky state outside the active window (files, notes, sub-agents) and returning only condensed results into the model’s context.
Early-exit rate
Product-facing label for rising premature termination on long agent runs: give-up and uncertain-incorrect endings before the window is exhausted. Used interchangeably with premature-termination rate in mitigation and measurement sections.
State trajectory
The evolving set of beliefs, tool results, and summaries a session has already accepted and reloads on later turns. Early wrong or thin context can survive compaction and still drive a bad ending.
Spine
The condensed set of beliefs, decisions, and claims a session treats as established. A false or thin spine can survive compaction even when length falls.
Distractor
Content that is topically related to the target information but does not answer the question. Often more harmful than unrelated filler.
Three clocks of rot
This paper’s framing: length (token count), pollution (distractors / near-misses), and state (accepted wrong or thin spine) can move independently. Fixing one clock can leave the others running.
Relative fullness (L_rel)
Input length divided by the model’s context window. Used to show how positional bias changes as the window fills.
Context debt
Low-signal tokens accepted into the live call: they dilute focus now and compound in later turns. Explanatory metaphor only; not a measured quantity.

1. Executive summary

Context rot is the measurable decline in an LLM’s output quality as input context grows, even when task difficulty is held fixed and the information required to succeed is still in the prompt. Marketed context limits describe how much text a model can accept. They do not describe how much it can use well. Treat that scarce resource as an attention budget.

What this paper argues

Rot has three clocks: how long the input is, how polluted it is, and what false or thin story the session has already accepted (accepted wrong state). Fixing length alone can leave noise and a bad spine (the session’s accepted story) untouched. The state trajectory is the second failure arc: early thin or wrong context can survive compaction, because length management is not truth management.

Stop packing long, noisy prompts before you compact, trim, or isolate. Remediation is for debt already incurred. Measure usable length as a portfolio of MECW bands by task family, not one global limit. For long-horizon agents, lead with early-exit and terminal-state rates against trajectory length, not exact match alone.

What the evidence shows

Position and relative fullness change what the model uses 4,9. Irrelevant and near-miss content hurt even when the answer is present 7,2. Strong needle-in-a-haystack scores overstate harder long-context work 3,5. Length can cut accuracy after perfect retrieval 1. Usable windows often sit far below marketed maxima by task type 6. Long-horizon agents show rising early-exit as trajectories grow 10. Self-attention cost scales with the square of sequence length, so the window is scarce, not free storage 8.

Three operating principles

  1. Prevent: treat context as an attention budget and send the smallest high-signal token set for the next decision.
  2. Remediate: compact, trim, and isolate when debt is already present; check state, not only fill. Do not treat the transcript as the system of record.
  3. Design: make rot harder to trigger by default with task-scoped sessions, just-in-time focused context, minimal tool returns, ephemeral transcripts, and observability as a ship requirement.

2. The problem in practice

Most people meet context rot as a feeling, not a benchmark. The assistant that was sharp at the start of a thread loses precision. An agent reports a fix that is still broken. A standing rule in the system prompt stops sticking. Teams reach for a larger context window or a newer model, and the same pattern returns a few dozen turns later.

There is a second arc that length control alone does not fix. Call that path the state trajectory: each turn reloads not only more tokens, but whatever beliefs, tool results, and summaries the session has already accepted. If thin, wrong, or incomplete context enters early, or if a false “fix” becomes part of history, compaction and trimming can shrink the window while preserving the error. The end outcome stays inaccurate because the spine (the session’s accepted story) was wrong. Length management is not truth management: a smaller window can still carry a wrong completion. Reliability needs a state check, not only a fill meter.

2.1 Failure modes users already recognize

These patterns show up across chat assistants, coding agents, and tool-using workflows. They are not proof of a bad prompt alone. They are what long, crowded, poorly managed context often looks like from the outside.

Standing instructions fade. Project rules, style guides, or safety constraints that worked early in a session get ignored later. As history grows, early material competes with newer turns and tool output. That pattern is consistent with positional research: middle content is easy to underuse, and primacy weakens as the window fills 4,9.

False completion loops. The model asserts that a bug is fixed, a file is updated, or a step is done. The user checks and finds no change, asks again, and receives another confident report. Each exchange adds tokens. The failure is now inside the window as “successful” history that can reinforce the next wrong move. That pattern is a practitioner observation consistent with later trajectory evidence; it is the state clock running while the length clock also ticks.

Contradiction across the thread. Mid-conversation answers conflict with earlier commitments or with documents still present in context. The information required to stay consistent may still be in the prompt. Length and distraction make consistent use harder 1,2.

Premature give-up or hedge. Especially in long agent runs, the model stops searching, declares the problem unsolvable, or answers with unresolved uncertainty that is still wrong. Give-up and uncertain-incorrect endings (premature termination) rise as search trajectories accumulate context 10.

Simple tasks get brittle in long sessions. Copying, following a short checklist, or retrieving a fact that was stated earlier should be easy. Controlled evaluations still show degradation on simple tasks as input length grows 2. That matches the practitioner experience of “this used to work ten messages ago.”

Practitioners have described the same arc in public writing about day-to-day LLM use: long chats degrade, and starting fresh often restores quality faster than arguing with the thread. Clearing or resetting removes accumulated length and pollution. It is not magic. It is a crude form of context management.

2.2 Why “buy a bigger window” misses the point

Vendor roadmaps keep raising MCW: 128K, 200K, 1M, and beyond. Those numbers matter for how much text you can submit. They do not equal how much text the model can use well.

Needle-in-a-haystack demos encourage the wrong conclusion. A model that can surface a planted sentence in a long haystack can still fail on harder long-context work, on conversational memory tasks, and on agent trajectories 3,5,2,10. Even perfect retrieval of the needed evidence does not prevent large accuracy drops as length grows 1. MECW measurements show the usable span for a given problem type is often a small fraction of the marketed maximum and moves with task difficulty 6.

So a bigger window can delay hard truncation and make some workflows possible. It does not cancel the attention budget. If the product’s default is to keep stuffing history, tools, and “maybe relevant” documents into every call, a larger MCW mostly delays the moment rot becomes obvious.

3. Foundations: what goes into context

Before context rot makes sense as a failure mode, the context window itself has to be concrete: what it is, what usually fills it on a single model call, why chat products make that fill grow automatically, and why it has a hard size in the first place.

3.1 The context window as short-term memory

The context window is the full set of tokens available to the model for one inference. In product language it is often compared to short-term or working memory: everything the model can condition on right now, not a durable archive of everything the user has ever said.

That metaphor is useful and incomplete. Human working memory is tiny and actively managed. An LLM window can hold tens or hundreds of thousands of tokens, which tempts teams to treat it like cheap storage. The better analogy is a large but lossy scratchpad. Capacity to accept text is not the same as capacity to use it evenly.

Window sizes differ by model and product tier. Public figures change often; treat any table of maxima as perishable. What does not change is the structure of the problem: every call has a finite window, and everything you put in that call competes for an attention budget inside it 8,4.

3.2 Anatomy of a typical call

Users often believe the only input is the latest message they typed. In almost every production system, the model receives a composite prompt assembled by the application. Common constituents include:

Component Role
System instructions Standing rules, persona, safety policy, output format
Tool and MCP schemas Names, descriptions, and parameter contracts for actions the model may call
Skills, plug-ins, or policy packs Extra instructions or capabilities loaded for the session
Retrieved documents RAG chunks, search hits, file excerpts attached for this turn
Conversation history Prior user and assistant messages the product chooses to resend
Tool results Raw or summarized outputs from the last actions (JSON, HTML, logs, code diffs)
The current user message The new ask

The object of design is this whole bundle: not only the wording of a system prompt, but which tokens are present at sampling time. Two products can use the same base model and behave very differently because they pack this bundle differently 4,2.

Figure 1: One call, many competitors
Figure 1. What competes inside one model call. Short inputs use tokens densely; long inputs thin effective focus.

Tool results deserve special attention. A single search page, spreadsheet export, or stack trace can dwarf the user’s actual question. Agents that append every observation to history grow the window fastest, which is exactly where give-up and uncertain-incorrect endings rise as trajectories lengthen 10.

3.3 Why chat interfaces make the window look infinite

Stateless chat is a fiction maintained for UX. To keep a conversation coherent, most chat-first applications resend prior turns (or a compacted form of them) on every request. The user experiences a continuous thread. The model experiences a longer and longer input.

Web and consumer UIs usually hide fill percentage, token accounting, and the invisible preamble (system text, tools, retrieval). The thread can feel unlimited because the product allows endless scrolling and never shows the budget. Under the hood, history still expands, older material drifts toward weaker positions as the sequence lengthens 4,9, and quality can fall long before any hard truncation error appears.

Figure 2: Chat turns manufacture longer context
Figure 2. Each turn reloads a larger window. Quality risk compounds with length and distractors until reset, compaction, or external handoff.

Some products eventually summarize, drop, or silently truncate. Those interventions are easy to miss from the user’s side. The result is a gap between mental model (“the assistant remembers this chat”) and mechanism (“the application is stuffing a finite window each turn”).

3.4 Why the window is finite

Three constraints bound context size in practice.

Compute. In the original transformer, self-attention scales with the square of sequence length 8. Longer prompts mean more latency and more spend per turn, which is why products and APIs meter tokens even when marketing headline windows look generous.

Memory. Serving long contexts requires holding large key/value caches and activations. Hardware and serving stacks impose real ceilings independent of marketing slides.

Training and position schemes. Models are typically pretrained and tuned with far more short sequences than million-token ones. Long-context adaptations (for example position interpolation and related extensions) widen the door without guaranteeing that long-range use will match short-context quality. That gap shows up empirically when benchmarks leave simple lexical retrieval behind 4,3. It is one reason MECW can sit far below MCW for hard tasks 6.

None of this requires a deep transformer tutorial. The product takeaway is simpler: the window is a scarce, expensive, unevenly trained resource. Designing as if it were free storage is how applications manufacture context rot.

4. Defining context rot

4.1 Working definition

Context rot is the measurable decline in an LLM’s output quality as input context grows, even when the task’s difficulty stays fixed and the information required to succeed remains in the input.

The underlying failure mode is older than the label. By 2023–2024 it was already clear that models do not reliably use information across long inputs, especially in the middle 4. Chroma’s 2025 technical report popularized the name context rot while evaluating 18 frontier and open-weight models under controlled length-only conditions 2. This paper uses “attention budget” for the practical stance those results imply: as token count rises, models use in-context information less reliably, so the window behaves like a finite resource with diminishing returns 8,4,1.

A large MCW only guarantees that tokens can be accepted. It does not guarantee they can be used well. Adding more context is never free. Extra tokens can dilute attention, shift positional biases, and interact with distractors, including text that looks harmless on inspection.

In practice, rot rarely arrives as a single hard failure at the token limit. Quality slips earlier: missed instructions, weaker retrieval, brittle reasoning, premature give-up, or rising error rates as inputs lengthen. How fast that slip appears depends on the model and the task. Length, pollution, and accepted wrong state can move on separate clocks; shortening the window can leave noise and a false spine untouched.

4.2 Nearby ideas that are not synonyms

Product teams often blur several related terms. Keeping them separate makes diagnosis and mitigation clearer.

Lost in the Middle is a positional finding. On multi-document QA and key-value retrieval, models often do best when relevant information sits at the beginning or end of the input, and worse when it sits in the middle, including for models sold as long-context 4. That pattern is real and useful. Context rot is wider. It also covers length-driven degradation that remains after position and retrieval are controlled.

Context pollution is about what is in the window. Even small amounts of irrelevant context can sharply cut reasoning accuracy 7. A useful split separates distractors (topically related but wrong) from irrelevant content (unrelated filler). Distractors often hurt more, and their impact grows unevenly with length 2. Pollution can accompany context rot. It is not required for it. Length-driven drops can remain when distractors are reduced to whitespace or, for some models, attention-masked away 1.

MECW versus marketed MCW is a measurement distinction. Paulsen defines the Maximum Effective Context Window as the longest useful span for a given problem type before additional tokens measurably hurt output. In those measurements, MECW diverged sharply from advertised MCW (in the worst reported comparisons by more than 99% on harder problem types) and shifted with task type 6. MECW is a way to locate where rot becomes intolerable for a workload. Context rot is the degradation MECW is trying to find.

Figure 3: Advertised max vs usable band
Figure 3. Schematic MCW versus MECW. Harder tasks shrink the usable band. Measure a portfolio of bands by task family; one global limit is the wrong kind of number. Illustrative only 6.

Hallucination is an outcome, not another name for rot. Unsupported fabrications can rise as contexts lengthen or exceed MECW 6. Rot also shows up as ignored rules, failed copying, give-up, or uncertain-incorrect endings without a classic invented fact 10,2.

4.3 A builder’s test

Hold the task fixed. Increase only input length or conversation accumulation. If success rate, reliability, or calibration worsens in a predictable way, you are seeing context rot, whatever the model’s advertised window says.

The evidence behind that test follows.

5. Evidence base

The findings below are grouped by what they establish, with primary sources attached. Read together, they show that context rot appears across model families; that “couldn’t find the needle” is an incomplete explanation; that position and window fullness matter; that simple lexical NIAH overstates competence; and that long-horizon agents exhibit their own rot signature.

5.1 Position: primacy, recency, and Lost in the Middle

Models do not treat every offset in a long input equally. In controlled multi-document question answering and key-value retrieval, performance was highest when relevant information appeared at the beginning or end of the context, and substantially worse when it appeared in the middle. Longer inputs made the problem worse 4.

For product and prompt design, that means critical instructions, policies, and evidence buried mid-transcript start at a structural disadvantage. The effect is not limited to one vendor.

5.2 How positional bias changes as the window fills

The familiar U-shaped Lost-in-the-Middle pattern is not stable across the full window. Measured against relative length Lrel (input length divided by the model’s context window), Lost-in-the-Middle intensity was strongest for Lrel ≤ 0.5. Past roughly half fullness, primacy fell sharply while recency held up better, leaving a distance-from-end bias: information closer to the latest tokens fares better 9.

That result reconciles earlier papers that did or did not report LiM. Studies on short relative lengths were more likely to see the U-shape. Studies that pushed near full long-context regimes were more likely to see it fade.

“Put the important material at the top” is therefore incomplete. Early system rules can lose force as a conversation fills the window. Re-asserting constraints late, or reducing fullness, becomes more valuable as occupancy rises.

Figure 4: Same fact, different place
Figure 4. Same fact, different place in the prompt. Under ~half full: start and end beat the middle (Lost in the Middle). As the window fills: early text fades and the latest turns win 4,9.

5.3 Length can hurt after retrieval succeeds

A common story says long-context failure is mostly retrieval failure. Direct tests on math, QA, and coding tasks across open- and closed-source models cut against that story. Even when models could recite all relevant evidence with exact match, task accuracy still fell substantially (reported range 13.9%–85%) as length grew inside claimed context limits 1. The drop remained when irrelevant text became whitespace, when evidence sat immediately before the question, and when distractor tokens were attention-masked so only evidence and question received attention (masking applies where the serving stack allows it, typically open-weight models) 1. Prompting the model to recite retrieved evidence and then solve on a short prompt raised GPT-4o on RULER by up to about 4% on an already strong baseline 1.

Figure 5: Length can hurt even when retrieval is perfect
Figure 5. Schematic: retrieval can remain intact while task success falls. Reported drops range 13.9%–85% by model and task 1.

Better RAG and better NIAH scores are not enough if the model still has to reason inside a very long window. Shortening the active reasoning context is a primary mitigation.

5.4 Controlled length-only tests on frontier models

Chroma’s Context Rot report ran 18 LLMs, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 families, through experiments that hold task difficulty steady while varying length: extended NIAH-style setups (needle–question similarity, distractors, haystack structure), LongMemEval conversational QA (full prompts averaging about 113K tokens versus focused subsets around 300 tokens), and a repeated-words replication task. Performance became less reliable as inputs grew, including on deliberately simple tasks 2. Many public long-context benchmarks mix length with difficulty. That design isolates length.

A larger window or a newer frontier model does not retire the problem. Uniform attention across 100K-plus tokens is an engineering assumption the data does not support.

5.5 Why needle-in-a-haystack is a weak yardstick

Near-perfect lexical NIAH scores are easy to over-read.

RULER adds multi-needle variants, multi-hop tracing, and aggregation on top of vanilla NIAH. Models with strong vanilla NIAH still dropped hard as length and task complexity rose 3. On the broader suite, only about half of evaluated systems stayed satisfactory at 32K 3.

NoLiMa reduces lexical overlap between questions and needles so models must rely on latent associations rather than string match. Across 13 models claiming at least 128K context, at 32K, 11 of those 13 fall below half of their short-context base score 5. GPT-4o, a relative exception, still drops from a 99.3% base score to 69.7% at 32K 5.

Marketing and internal evals that lean on NIAH alone will under-detect rot. Stronger suites (RULER, NoLiMa, and agent trajectory metrics) are closer to the failure modes products actually ship.

5.6 Effective context versus advertised context

Large-scale MECW measurements across prominent models and problem types find usable windows often far below marketed maxima 6. Some top models failed near 100 tokens on certain tasks. Many showed severe accuracy loss by about 1,000 tokens on harder types. Gaps versus advertised maxima reached more than 99% in the worst reported comparisons 6. Model rankings also changed across task types 6. Practical counsel: size agent and task scopes to MECW rather than MCW, and split failing agents into narrower sub-tasks when needed.

“We have a 1M-context model” is a capacity claim, not an architecture. Task design, chunking, and agent scope decide whether that capacity helps.

This paper’s durable product takeaway is a MECW portfolio, not a single number. Easy retrieval, multi-hop research, and long coding sessions usually need different usable bands. Ship those bands, route work into the right one, and split failing jobs before buying another marketed maximum. One global context limit is a category error. Size and route from task-dependent measurements; do not treat published tables as a portable SLA 6.

Treat the reported magnitudes as directional evidence that should push you to measure your own workloads 6.

5.7 Long-horizon agents: premature termination

Single-shot retrieval misses are only part of the agent story. Deep search benchmarks with a terminal-state taxonomy (give up, uncertain answer correct or incorrect, confident answer, and no answer) show give-up and uncertain-incorrect endings rising under growing context—premature termination: models stop or hedge long before the window is exhausted, and that combined rate rises with context length when query difficulty is controlled. Context management (compaction, trimming, isolation) and behavior-aware filtering before parallel aggregation are evaluated as mitigations 10.

Agents that append every tool dump into one window inherit multi-turn rot by construction. Mitigations have to manage trajectories. For measurement, treat early-exit and terminal-state rates against trajectory length as the primary long-horizon KPI; exact match alone understates the failure. This paper’s playbook puts prevention first, then compaction, trimming, and isolation for debt already incurred (mid-flight methods evaluated for long-horizon search 10).

5.8 From evidence to operating stance

Taken together, the studies above support treating context as scarce: send the smallest high-signal token set that still produces the desired behavior.

Limits of the literature

A few cautions belong next to the citations.

  • Strong lexical retrieval underestimates failure on semantic, multi-hop, aggregative, and agentic work 3,5,2.
  • Degradation slopes, refusals, and distractor sensitivity vary by model family; averages hide product-relevant variance 2,1.
  • Archival anchors (Vaswani, Shi, Liu, Du, RULER, NoLiMa, Paulsen) sit on firmer ground than lab reports and early preprints. Chroma is a technical report that names and extends the length-degradation finding; it does not originate the scarcity claim. Veseli et al. and Xia et al. remain arXiv / preprint-stage here; use them for mechanisms and hypotheses, then replicate on your stack before treating numbers as SLAs.
  • Position and length effects transfer from long documents to chat and agents, but early-exit dynamics and trajectory noise need agent-specific evaluation 10.
  • No study in this set claims a larger window eliminates context rot. Larger MCWs delay truncation. They do not erase the attention budget.

6. Mechanisms of context rot

The previous section established the empirical pattern. The mechanisms below are the main reasons that pattern keeps appearing. They stack. Architecture, training data, position encoding, content quality, and multi-turn accumulation can all push in the same direction.

6.0 Three clocks of rot

It helps to separate three clocks that can run independently:

Clock What moves Typical evidence
Length Token count and relative fullness Accuracy falls as inputs grow even after retrieval succeeds 1,2
Pollution Distractors and near-miss packing Related wrong text hurts more than empty filler 7,2
State Beliefs, tool results, and false completions already accepted Trajectories accumulate give-up and wrong endings; a bad spine can survive shortening 10

Fixing one clock can leave the others running. Compacting a polluted trajectory can cut length while preserving a false completion. Filtering distractors can leave a call that is still too long to reason in. Clearing history can drop pollution and length while discarding needed decisions if durable state was never stored outside the window.

Figure 6: Three clocks of rot
Figure 6. Length, pollution, and state move on separate clocks. Product moves should name which clock they change.

The subsections below unpack the architecture and training reasons those clocks keep ticking.

6.1 Attention behaves like working memory, not disk storage

Transformers let tokens attend across the input. In the original formulation, self-attention computes relationships among tokens with cost that grows as O(n2) in sequence length 8. As length grows, those relationships get harder to keep sharp in practice. Each new token competes for a finite attention budget, and the value of stuffing more text falls 4,1.

A disk either stores a file or it does not. An attention budget can accept more tokens while giving each one less effective focus. MCW answers how much can be accepted. MECW and task success answer how much can still be used well.

Perfect-retrieval drops fit this picture. The failure is often in using evidence inside a long input, not only in locating it 1.

6.2 Training data skews short

Attention patterns come from training distributions that contain far more short sequences than very long ones. Models therefore have less experience, and fewer specialized parameters, for dependencies that span an entire long context. That skew is a standing explanation for why long-context quality lags advertised reach 4,3.

Behavior that looks stable at a few thousand tokens can weaken when sessions, RAG packs, or tool traces enter length regimes that were rare in training, even when those lengths sit under the advertised MCW. Part of context rot is distribution shift.

6.3 Long windows strain position signals

Many systems extend beyond their original training lengths with position-encoding interpolation and related adaptations. Reach increases. Precision of token-position representation can soften, which is consistent with positional failures that intensify as inputs grow 4,9.

That interacts directly with the positional results above: primacy, recency, and the move from Lost-in-the-Middle toward distance-from-end bias as relative fullness grows 4,9. If position signals are already under strain, mid-transcript instructions and early rules that have drifted far from the generation point become easier to miss.

Models can remain useful at long contexts while losing precision on retrieval and long-range reasoning 4,1,2. The decline is gradual more often than catastrophic.

6.4 Distractors cost more than empty filler

Length is not the only variable. Irrelevant and near-miss content cut accuracy even when the correct evidence is present; related wrong text often hurts more than empty filler, while length-driven drops can remain even after distractors are reduced 7,2,1.

Fill type Typical effect
Near-duplicate or misleading related text High confusion; competes with the true answer
Unrelated but coherent filler Still costs attention; can change structure effects
Near-empty padding (e.g., whitespace) Lower semantic distraction; length effects can remain

Dumping “maybe useful” CRM notes, prior tool JSON, or similar tickets into the window is often worse than dumping random noise of the same length. Near-miss content is a distractor.

Call that accumulated low-signal material context debt. Extra tokens dilute the call now (length and pollution clocks) and compound later when the same material is resent as history (state clock). The metaphor is explanatory only. It is not a measured quantity. Prefer naming which clock you are changing when you cut debt.

6.5 Multi-turn accumulation differs from one long document

Most classic long-context papers study a single long input. Chat UIs and agents usually build context turn by turn. Premature termination rises as trajectories lengthen under controlled difficulty; aggressive clearing reduces those endings and can leave more work unfinished if durable state was never stored outside the window 10.

Production trajectories differ from clean essay haystacks: tool dumps, HTML, logs, retries, and the model’s own earlier mistakes stay in context. Recency tracks the latest turns, so early system rules drift into a primacy-fragile zone as the window fills 9. That is the state trajectory: remediation that only shortens the window can still replay a flawed trajectory. Length management is not truth management.

Chat-first and agent products do not merely possess long context. Unless they compact, trim, or isolate, and unless they stop packing every candidate into every call, they keep manufacturing the conditions measured in the evidence base.

6.6 What builders should optimize against

Mechanism Effect Design pressure
Diluted pairwise attention More tokens, thinner focus per token Prefer fewer, higher-signal tokens
Short-sequence training bias Long regimes are out of distribution Do not treat MCW as reliable length
Stretched position encodings Weaker grip on where facts sit Re-assert critical rules; manage fullness
Distractors versus filler Near-miss content hurts more than empty padding Filter for relevance, not volume
Multi-turn accumulation History and tools grow a hostile window over time; early state compounds Compact, trim, or isolate; seed the right spine early
Three clocks out of sync Length, pollution, and state move separately Name which clock a change is meant to fix

These mechanisms are the ordinary cost of asking a fixed attention architecture to act like unbounded memory. The following sections cover where that cost shows up in products, then how to reduce it.

7. Implications by use case

Context rot is a model-level phenomenon. Products decide how often users hit it. The same attention budget produces different failure shapes in consumer chat, coding agents, RAG systems, and deep research loops.

7.1 Consumer chat UIs

Chat-first products are where most people first notice rot, because they optimize for continuity. Prior turns are resent (or lightly compacted) so the thread feels like one ongoing mind. System text, tools, and retrieval often sit outside the visible transcript. Fill percentage is rarely shown.

Under that design, the user’s main levers are blunt: start a new chat, or ask for a summary and paste it into a fresh thread. Both shrink length and drop accumulated distractors. They also discard useful state the product never stored anywhere else.

When a consumer chat product markets “memory” without separating durable facts from the live window, it can worsen the problem. Recalled notes tossed into an already long thread are more tokens competing for attention, not a free upgrade to permanence. LongMemEval-style conversational settings already show frontier models degrading when relevant answers must be found inside large chat histories 2.

7.2 Coding agents and long sessions

Coding agents combine chat history with tool results: file reads, diffs, test output, linter noise, terminal logs. A single failed test run can dominate the next call. False-completion loops are common here because “I fixed it” becomes part of the trajectory the model conditions on next.

Session length interacts with task hardness. Planning, multi-file refactors, and debugging across uncertain root causes have lower effective windows than narrow edits 6. Teams that keep one epic thread for an entire feature often watch quality fall exactly when the work gets hardest.

Agents with visible context controls (inspect, compact, clear) give operators a chance to intervene. Agents that hide the budget behave like consumer chat with heavier per-turn load: more tools, more debris per turn, faster accumulation 10.

Retrieval-augmented generation is often sold as the cure for limited context: fetch the right passages, then generate. That helps when retrieval is precise and the packed prompt stays short. It hurts when systems retrieve generously “to be safe.”

Topically related near-misses are distractors. Stuffing ten similar policy paragraphs raises length and confusion together 2. When success falls as near-miss chunk counts rise, retrieval volume is an anti-metric: a liability score, not a quality score 2,7. Lexical NIAH success does not guarantee the model can use retrieved evidence under weak question–passage overlap 5. Even perfect evidence can fail if reasoning still happens inside a long padded prompt 1.

Enterprise deployments that concatenate large ticket histories, CRM dumps, and “related documents” into every answer request are running a retrieval pipeline that still manufactures rot. RAG-style packing can also push hallucination rates up once token counts exceed task-specific MECW—a warning for those pipelines 6.

Long-horizon search agents accumulate environment feedback: SERP snippets, page extracts, intermediate hypotheses, retries. As that material grows under controlled query difficulty, premature termination rises: more give-up and uncertain-incorrect endings before the window is exhausted 10.

Multi-hop work also stresses aggregation and tracing, where RULER already finds large drops relative to simple lexical retrieval 3. The practical consequence is architectural. A single agent that keeps every page visit in one window will hit rot by construction. Systems that isolate exploration in sub-agents, return condensed evidence, or stage recite-then-reason steps follow from the same length and distractor evidence.

7.5 Product design: when “remember everything” hurts reliability

“Remember everything” is a strong marketing line and a weak reliability strategy. Unlimited conversational memory, maximal RAG, and full tool transcripts optimize for completeness of input, not quality of inference.

Design choices that raise rot exposure:

  • Defaulting to one unbounded thread per user or per project
  • Hiding token budget and composition from operators
  • Treating retrieval volume as a quality proxy (it is often an anti-metric)
  • Leaving raw tool payloads in history after their useful life
  • Placing critical policies only at the start of a prompt that will grow for hours

The through-line is simple. Products that treat the transcript as the system of record push the attention budget until it breaks. Products that treat the window as a temporary workspace for the next decision have a chance to stay reliable as work gets longer.

8. Mitigation framework

Context rot is not fixed by a larger MCW. It can be reduced by shrinking what enters the window, deciding when to rewrite or drop history, and designing products so the default path does not manufacture long, noisy prompts.

Order the work as prevent → remediate → design. Stop manufacturing long, noisy prompts before you compact, trim, or isolate. Mid-flight remediation is for debt already incurred. Product-shape defaults keep the next session from recreating that debt. Compaction, trimming, and isolation are mid-flight methods evaluated for long-horizon search 10; the prevention-first hierarchy and the MECW-portfolio routing around them are this paper’s synthesis on the length, pollution, and trajectory evidence.

None of these moves “solve” the attention budget. They keep more work inside the span where models still behave.

8.1 What operators can do today

When the product exposes few controls, human workflow is still the mitigation layer.

Start a new thread when the topic changes, when the model starts looping, or when the conversation has grown long enough that quality is slipping. Ask for a short handoff summary first if you need continuity, then paste only that summary into the fresh session. Keep pasted material tight: the goal, constraints, and current artifacts, not the full argument history.

Prefer pointers over dumps. Link or name the file, ticket, or doc instead of pasting entire bodies when the product can fetch them. Restate critical rules near the point of use if an early system note has stopped sticking. Primacy weakens as relative fullness rises 9.

Split hard work. If a session mixes planning, research, and execution, end the planning thread once the plan is written down outside the chat, then run execution in a cleaner window. Usable span tracks problem type more than marketing copy 6.

8.2 Trajectory management

Prevent (before remediation)

See the budget. Expose how full the window is, what categories of tokens dominate (system, tools, history, retrieval, tool results), and simple failure signals (repeated claims, early-exit phrasing, contradiction with pinned rules). Consumer chat that hides fill leaves users with only the blunt reset operators use today. Observability is governance for measurement, not a fourth remediation family.

Send less, fetch on demand. Before compacting a bloated trajectory, stop manufacturing one. Length alone can cut accuracy after retrieval succeeds 1. Irrelevant and near-miss content distracts even when the answer is present 7,2. Positional underuse worsens as inputs grow 4. The operational consequence: assemble the smallest high-signal set for the next call; keep system text short enough to enforce behavior; ship few tools with clear contracts; prefer refetchable pointers over encyclopedic dumps. Packing every “maybe useful” candidate up front recreates the distractor and length conditions shown in the evidence and adds context debt.

Fit tasks to the MECW portfolio. Measure usable length for your problem types, keep each call inside its band, and route hard work into a narrower stage before buying another marketed maximum 6. Scope is a reliability control.

Place and re-assert with fullness in mind. Critical constraints at only the head of a prompt that will grow for hours fight the primacy fade that sets in as relative fullness rises 9. Re-pin rules near the end when occupancy is high, or reduce occupancy so early instructions stay effective. Do not rely on the Lost-in-the-Middle pattern alone; relative fullness shifts where that bias holds.

Relative fullness First product moves
About half full or less Placement still matters: keep critical evidence out of the deep middle; test head / middle / tail 4,9
Past about half full Re-pin standing rules near the recent end, protect recency, or cut fill before trusting an early prefix alone 9

Recite, then reason. After retrieval, have the model extract the evidence it will use into a short block and answer from that block rather than from the full haystack 1. The point is to convert a long-context solve into a short-context solve once the needles are known.

Remediate (debt already incurred)

Once a session is already long, the controllable moves are compaction, trimming, and isolation 10. Make them measurable, preferably automatic, and visible enough that operators can see what is filling the window. After any rewrite or drop, run a state check: does the retained spine still match ground truth and pinned rules? Length management is not truth management.

Compact. When history approaches a threshold, summarize decisions, open questions, and artifacts into a shorter replacement context, then continue 10. Compaction is lossy. Tune for recall of what still matters, then cut redundancy. Clearing stale tool results is a light form that often costs little. Compaction changes the length clock; it does not by itself audit whether the retained spine is true.

Trim. Drop rather than rewrite. Discard old observations, duplicate retrieval hits, and superseded plans while keeping IDs or file paths so the agent can refetch 10. Trimming pairs well with compaction: rewrite the spine, delete the bulk. Prefer dropping false or superseded claims, not only old tokens.

Isolate. Keep bulky state outside the active window. Notes, plans, and scratch files on disk; memory stores; sub-agents that explore and return a short brief 10. Isolation is how long-horizon work stays possible without one trajectory absorbing every page visit and log line.

Those three are the mid-flight spine. Do not ship on vanilla NIAH alone.

Choosing a move under pressure

Situation First moves
Web chat, no visibility New thread; summarize-and-restart; stop pasting whole docs
Quality falling mid-run Inspect composition; trim tool dumps; compact; clear and reload plan from disk; audit retained spine
RAG answers drift or contradict Fewer chunks; higher relevance bar; recite-then-reason; measure MECW for the question type
Deep research / multi-hop Isolate browsing in sub-agents; return briefs; cap trajectory length before early-exit rates spike
Hard planning + execution Separate sessions; persist the plan outside the model; execute in a short window

Context management reduces premature termination, and the best mix is model-dependent: isolation via sub-agents can lead when the backbone is strong, while compaction plus trimming often balances performance and cost otherwise; more frequent passive management cuts rot at higher spend 10. Treat that as guidance to validate on your stack, not as a universal ranking.

8.3 Patterns that reduce rot by construction

Operator discipline and mid-flight compaction help. Product shape matters more. Some designs expose users to rot on every long task. Others keep the live window short because context arrives only when needed and durable state lives elsewhere. The patterns below are common ways products keep the live window short—not another mid-flight family beside compact / trim / isolate.

Just-in-time focused context. Pull only what the next decision needs (for example the document or file under consideration) instead of asking the user to keep everything in a chat archive. The model sees a small, relevant slice when it acts. That is the product form of “fetch on demand” from prevent / fetch-on-demand: do not pre-stuff every candidate corpus into the call 7,2,1.

Task-scoped sessions. Prefer a clean window per job over one eternal thread per user. Carry forward a plan or checklist stored outside the model, not the full argument history. That keeps each call closer to the task’s MECW 6.

External environment. Persist goals, decisions, and artifacts outside the model call. Pull summaries back when required. This is isolation as product default, not only as an in-flight recovery tool 10.

Actions that return minimal payloads. Tools should give the next decision what it needs, not a full export every time. Bloated schemas and verbose results manufacture length and distractors without buying reliability 7,2.

Treat the transcript as ephemeral. Do not rely on the full chat history as the durable record of goals, decisions, or outputs. Persist those outside the model call and reload only what the next turn needs. That cuts multi-turn accumulation before it compounds in later turns.

These patterns do not remove transformer limits. They reduce how often products force models to reason inside long, polluted windows. Whether that reduction holds in production is a measurement problem.

Figure 7: Keep the live window short
Figure 7. Prevent (see the budget; send less / fetch on demand) → remediate with compact / trim / isolate if needed → call. Solid path is mid-flight; dashed foundation is product-shape defaults.

9. Measurement and governance

Mitigations without measurement become anecdotes. Teams need curves, not slogans: how quality moves with length, where position breaks, when agents give up, and what management costs in tokens and latency. Governance then turns those numbers into product requirements and operating thresholds.

9.1 Metrics that detect rot

Accuracy (or task success) versus input length. Hold the task fixed and vary only length, the way controlled length-only evals argue long-context tests should 2. Plot success against token count for each major workload (support answer, code edit, multi-hop research). The slope is the rot signal. A flat NIAH line with a falling RULER or NoLiMa line means you have been under-testing 3,5.

Position sweeps at fixed length. Place the same evidence at head, middle, and tail (and, if useful, as a fraction of Lrel). Position studies show why a single “needle at 50%” number is incomplete 4,9. Track primacy and recency separately as fullness changes.

MECW portfolio by problem type. For each production task family, find the length band where quality remains acceptable, treat that band as an operational limit, and route work into the matching band 6. Re-measure when models, prompts, or retrieval stacks change. Copying another team’s single number is unsafe.

Agent terminal-state rates (primary long-horizon KPI). For long-horizon agents, lead with give-up, uncertain-incorrect, confident wrong, and no-answer rates against trajectory length 10. Exact match alone understates the failure; rising early-exit rates with length are rot even when final-string accuracy looks noisy. Stronger suites than lexical NIAH already warn against under-testing 3,5.

Retrieval volume as an anti-metric. Compare packed prompts with high-similarity near-misses versus tight retrieval sets 2,7. If “helpful” extra chunks lower success, chunk count is a liability score, not a quality score.

Cost and latency versus management. Compaction, sub-agents, and frequent trimming spend tokens to save reliability 10. Report dollars and p95 latency next to rot metrics so teams do not optimize one while ignoring the other.

Composition share. Break each call into system, tools/schemas, history, retrieval, and tool results. Products that cannot name what dominates the window cannot govern it.

9.2 When to compact, trim, or clear

Thresholds should be empirical, not copied from another product’s status bar. A workable process:

  1. Build length–quality curves for the two or three task types that matter most.
  2. Mark an operating band. Pick the drop in success you will not tolerate for that task type (call it X%), then stay below the length where success falls more than X% from the short-context baseline.
  3. Trigger trim when tool results or retrieval dominate composition and are older than the current subtask.
  4. Trigger compact when history is large but still contains decisions the next call needs.
  5. Trigger clear (or a new session with an external handoff) when compaction would destroy more signal than it keeps, or when false-completion loops appear (state check, not only fill).

Harder tasks usually need earlier intervention than simple ones 6. Firing passive management more often reduces rot and raises cost 10. Governance is choosing that tradeoff explicitly: which workflows pay for aggressive management, which tolerate more risk.

Relative fullness also matters for what you protect. As occupancy passes the regime where primacy fades, re-pin critical rules or reduce fill before relying on an early project-rules prefix alone 9. Use the fullness table above.

Avoid a single global “compact at 80%” rule without task splits. That mistake is the MECW-portfolio error in threshold form: eighty percent of a 1M window on a hard reasoning job may already be deep into failure; twenty percent on a narrow classify-and-route job may be fine.

9.3 Product requirements: observability as governance

If context is an attention budget, observability is financial reporting for that budget. Treat it as a ship requirement for any chat or agent surface that runs long sessions.

Minimum product bar:

  • Show fill level (tokens or percent of the active limit) during the session.
  • Show composition breakdown on demand (system, tools, history, retrieval, results).
  • Offer compact, trim, and clear (or equivalent) as user-visible or policy-driven actions.
  • Persist durable state outside the live window when the product claims memory.
  • Emit logs suitable for the rot metrics (length, position tests in CI, agent terminal states, retrieval chunk counts).
  • Gate releases on rot-relevant evals, not only short-prompt golden sets and lexical NIAH.

Organizational bar:

  • Name owners for context policy (usually platform or applied-AI eng, with product sharing UX for visibility).
  • Review MECW and rot curves when model routers or default prompt packs change.
  • Include context cost and failure modes in incident reviews when assistants “got worse” mid-thread.

Without those controls, the playbook stays advice for power users. With them, context management becomes part of how the company ships and operates LLM features.

10. Recommendations

Twelve practices under Prevent / Remediate / Design. Each pairs a default to adopt with a default to refuse. They apply to individual operators and to teams shipping LLM products.

Prevent (1–6)

  1. Prevent first: treat context as a budget, not a warehouse.

    Do send the smallest high-signal token set for the next call 4,1,7.

    Do not equate a larger MCW with permission to stuff more by default.

  2. Optimize for an MECW portfolio, not marketed length.

    Do measure usable bands per problem type and route work into the right band 6.

    Do not publish or plan as if one global MCW were usable end to end 6.

  3. Hold task difficulty fixed when you evaluate long context.

    Do run length sweeps and harder suites (RULER, NoLiMa-class, conversational memory, agent trajectories) 2,3,5,10.

    Do not ship on lexical NIAH alone 3,5,2.

  4. Assume position and fullness change what the model uses.

    Do re-pin critical rules as sessions grow, and test head/middle/tail placement 4,9.

    Do not bury standing policy only at the start of an unbounded thread.

  5. Prefer fewer relevant tokens over more “maybe useful” tokens.

    Do raise the bar for RAG and tool payloads; related near-misses are distractors 2.

    Do not treat retrieval volume as a quality score; treat rising near-miss counts as an anti-metric when success falls 2,7.

  6. Shorten the reasoning window after you have the evidence.

    Do recite-then-reason or otherwise solve on a compact evidence block 1.

    Do not leave the model to reason inside the full haystack once the needles are known 1.

Remediate (7–8)

  1. Remediate trajectories, and check state.

    Do compact, trim, and isolate as first-class behaviors in agents 10.

    Do not treat a smaller window as proof the spine is true; do not append every tool dump to an immortal history 10.

  2. Store durable state outside the live window.

    Do keep plans, decisions, and artifacts in files, notes, or memory stores; pull summaries back as needed.

    Do not use the chat transcript as the system of record.

Design (9–12)

  1. Scope sessions to tasks.

    Do start clean windows for new goals; hand off with a short external brief.

    Do not run planning, research, and execution as one endless thread when quality is already slipping.

  2. Make the budget visible.

    Do show fill level and composition, and expose compact/clear (or policy automation).

    Do not hide context accounting in products that invite long sessions.

  3. Pay for management where reliability matters.

    Do accept higher token cost for compaction or sub-agents on high-stakes flows 10.

    Do not assume the cheapest path (never remediate context) is the reliable path 10.

  4. Design products so rot is harder to trigger by default.

    Do favor just-in-time focused context, task-scoped sessions, minimal tool returns, and ephemeral transcripts.

    Do not market “remember everything” as a reliability feature.

11. Open questions and future work

The evidence for context rot is strong enough to change how products are built. It is not finished science. Several gaps will shape the next round of research and engineering.

11.1 Will better models retire the attention budget?

Frontier systems keep improving on long-context benchmarks, and degradation slopes differ by family 2. It is fair to ask whether rot is a temporary artifact of today’s training mix and position schemes.

Nothing in the current literature says larger MCWs or stronger models remove the need to send less and manage trajectories. Self-attention’s quadratic growth with length is architectural 8. Empirical curves still show length-driven drops after retrieval is solved 1, positional underuse on long inputs 4, and frontier variance under controlled growth 2. A live research question is how far architectural and training advances flatten those curves, and which failure modes (positional bias, distractor fragility, premature termination) move together versus separately. Until products re-measure on their own workloads, “the next model will fix it” is a hope, not a plan.

11.2 Benchmarks for multi-turn and agentic rot

Most classic suites still stress a single long input. Production pain is cumulative: chat history, tool junk, self-generated error, and shifting recency 10.

The field needs shared, reproducible agent and multi-turn benchmarks that:

  • Hold underlying task difficulty steady while trajectories lengthen
  • Report give-up, uncertain-incorrect, and confident-wrong rates, not only exact match
  • Separate content effects (what accumulated) from turn-count effects
  • Compare compaction, trimming, and isolation under the same protocol 10

Consumer chat needs the same honesty: conversational memory evals at realistic fill levels, with and without product-side memory injection 2. Without those tests, vendors can look strong on NIAH while shipping threads that rot in the wild.

11.3 Learned versus heuristic context management

Today’s mitigations are mostly policies: summarize at a threshold, drop old tool results, spawn a sub-agent, recite then answer. Post-training that teaches models to edit their own context, or routers that choose management actions, is an active line of related work 10.

Open questions include whether learned managers generalize across tasks, how they fail under distribution shift, and how to evaluate them without hiding rot behind opaque compression. Heuristics remain auditable. Learned systems may win on cost and quality, but they need the measurement metrics or teams will not know what they traded away.

11.4 Measurement standards for MECW

MECW as a construct is practically useful; measurement standards are still maturing 6. The community still lacks a consensus protocol: which task families, how to define the failure point, how to compare models fairly, how to publish uncertainty. Until that matures, each organization should treat internal MECW as a living control limit, not as a portable scoreboard number.

11.5 Product research: defaults and affordances

Technical mitigations only help if people use them. We still know too little about which UX patterns change behavior: always-on fill meters, automatic compaction with undo, forced task-scoped sessions, or agents that refuse to continue until state is externalized. Studies that tie interface defaults to rot metrics would close the loop between product design, mitigation, and measurement.

11.6 What not to wait for

Teams do not need settled answers to the open questions below before acting. Length-controlled degradation, positional effects, weak NIAH transfer, and premature termination under accumulation are already documented well enough to justify observability, tight context selection, trajectory management, and product design that prevents rot by default. Future models and benchmarks should update the curves. They should not postpone governing the budget.

References

Numbered citations in the text map to this list.

  1. [1] Du, Y., Tian, M., Ronanki, S., Rongali, S., Bodapati, S., Galstyan, A., Wells, A., Schwartz, R., Huerta, E. A., & Peng, H. (2025). Context length alone hurts LLM performance despite perfect retrieval. Findings of EMNLP 2025. https://aclanthology.org/2025.findings-emnlp.1264/ · https://arxiv.org/abs/2510.05381
  2. [2] Hong, K., Troynikov, A., & Huber, J. (2025, July 14). Context rot: How increasing input tokens impacts LLM performance (Technical report). Chroma. https://www.trychroma.com/research/context-rot
  3. [3] Hsieh, C.-P., Sun, S., Kriman, S., Acharya, S., Rekesh, D., Jia, F., & Ginsburg, B. (2024). RULER: What’s the real context size of your long-context language models? Conference on Language Modeling (COLM 2024). https://openreview.net/forum?id=kIoBbc76Sy · https://arxiv.org/abs/2404.06654
  4. [4] Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://aclanthology.org/2024.tacl-1.9/
  5. [5] Modarressi, A., Deilamsalehy, H., Dernoncourt, F., Bui, T., Rossi, R. A., Yoon, S., & Schütze, H. (2025). NoLiMa: Long-context evaluation beyond literal matching. Proceedings of ICML 2025. https://proceedings.mlr.press/v267/modarressi25a.html · https://arxiv.org/abs/2502.05167
  6. [6] Paulsen, N. (2026). Context is what you need: The maximum effective context window for real world limits of LLMs. Advances in Artificial Intelligence and Machine Learning, 6(1). https://doi.org/10.54364/AAIML.2026.61268 · arXiv:2509.21361. https://arxiv.org/abs/2509.21361
  7. [7] Shi, F., Chen, X., Misra, K., Scales, N., Dohan, D., Chi, E., Schärli, N., & Zhou, D. (2023). Large language models can be easily distracted by irrelevant context. Proceedings of ICML 2023. https://proceedings.mlr.press/v202/shi23a.html · https://arxiv.org/abs/2302.00093
  8. [8] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. https://arxiv.org/abs/1706.03762
  9. [9] Veseli, B., Chibane, J., Toneva, M., & Koller, A. (2025). Positional biases shift as inputs approach context window limits. arXiv:2508.07479. https://arxiv.org/abs/2508.07479
  10. [10] Xia, S., Wang, Y., Huang, Z., & Liu, P. (2026). Diagnosing and mitigating context rot in long-horizon search. arXiv preprint arXiv:2606.29718. https://arxiv.org/abs/2606.29718 (v2; first posted 29 June 2026, revised 4 August 2026; code: https://github.com/GAIR-NLP/ContextRot)