Contextual Drag: How Errors in the Context Affect LLM Reasoning
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Central to many self-improvement pipelines for large language models (LLMs) is the assumption that models can improve by reflecting on past mistakes. We study a phenomenon termed contextual drag: the presence of failed attempts in the context biases subsequent generations toward structurally similar errors. Across evaluations of 11 proprietary and open-weight models on 8 reasoning tasks, contextual drag induces 10-20% performance drops, and iterative self-refinement in models with severe contextual drag can collapse into self-deterioration. Structural analysis using tree edit distance reveals that subsequent reasoning trajectories inherit structurally similar error patterns from the context. We demonstrate that neither external feedback nor successful self-verification suffices to eliminate this effect. While mitigation strategies such as fallback-behavior fine-tuning and context denoising yield partial improvements, they fail to fully restore baseline performance, positioning contextual drag as a persistent failure mode in current reasoning architectures.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 36 of its 47 references that are in the map. First posted on arXiv 4 Feb 2026.
Builds on
36 of 47 references.
Closest ancestors · 8
Same direction, newest first
- 2025Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning2 Dec
- 2025Rethinking Thinking Tokens: LLMs as Improvement Operators1 Oct
- 2025Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models30 Sep
- 2025gpt-oss-120b & gpt-oss-20b Model Card8 Aug
- 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?2 Feb
- 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement18 Feb
- 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
- 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
Other references · 28
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
- 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
16 more
- 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
- 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
- 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 165
- 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 106
- 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Modelscited by 104
- 2024Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Modelscited by 87
- 2023Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framingcited by 49
- 2023Resolving Knowledge Conflicts in Large Language Modelscited by 49
- 2024Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?cited by 40
- 2024Anchoring Bias in Large Language Models: An Experimental Studycited by 37
- 2025Taming Knowledge Conflicts in Language Modelscited by 27
- 2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critiquecited by 24
- 2025What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoTcited by 24
- 2025Patterns Over Principles: The Fragility of Inductive Reasoning in LLMs under Noisy Observationscited by 12
- 2026Lost in the Noise: How Reasoning Models Fail with Contextual Distractorscited by 8
- 2025Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigationscited by 6
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026RLTL;DR: Self-improvement by Internalizing Self-generated Feedback29 Sep 2026
- 2026Self-Compacting Language Model Agents22 Jun 2026
- 2026Old Habits Die Hard: How Conversational History Geometrically Traps LLMs8 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2026Reasoning Shift: How Context Silently Shortens LLM Reasoning
- 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
- 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
- 2024Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems
- 2026Dissecting Failure Dynamics in Large Language Model Reasoning
- 2024Internal Consistency and Self-Feedback in Large Language Models: A Survey
- 2026How Language Models Fail: Token-Level Signatures of Committed and Persistent Reasoning Failures
- 2026"I May Not Have Articulated Myself Clearly": Diagnosing Dynamic Instability in LLM Reasoning at Inference Time