mlsanityData to 7 Oct 2026

Context Length Alone Hurts LLM Performance Despite Perfect Retrieval

arXiv 2510.05381 ↗First posted 6 Oct 2025cs.CL cs.AIcited by 118

Not on the map. Why?

Cited by 41 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) often fail to scale their performance on long-context tasks performance in line with the context lengths they support. This gap is commonly attributed to retrieval failures -- the models' inability to identify relevant information in the long inputs. Accordingly, recent efforts often focus on evaluating and improving LLMs' retrieval performance: if retrieval is perfect, a model should, in principle, perform just as well on a long input as it does on a short one -- or should it? This paper presents findings that the answer to this question may be negative. Our systematic experiments across 5 open- and closed-source LLMs on math, question answering, and coding tasks reveal that, even when models can perfectly retrieve all relevant information, their performance still degrades substantially (13.9%--85%) as input length increases but remains well within the models' claimed lengths. This failure occurs even when the irrelevant tokens are replaced with minimally distracting whitespace, and, more surprisingly, when they are all masked and the models are forced to attend only to the relevant tokens. A similar performance drop is observed when all relevant evidence is placed immediately before the question. Our findings reveal a previously-unrealized limitation: the sheer length of the input alone can hurt LLM performance, independent of retrieval quality and without any distraction. They motivate our simple, model-agnostic mitigation strategy that transforms a long-context task into a short-context one by prompting the model to recite the retrieved evidence before attempting to solve the problem. On RULER, we observe a consistent improvement of GPT-4o up to 4% on an already strong baseline.

Key facts. Cited by 118 papers in the map (data through 7 Oct 2026). Builds on 42 of its 57 references that are in the map. First posted on arXiv 6 Oct 2025.

Builds on

42 of 57 references.

References · 42

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  7. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  8. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 360
  9. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  10. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 219
  11. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
  12. 2023Landmark Attention: Random-Access Infinite Context Length for Transformerscited by 206
30 more
  1. 2024Data Engineering for Scaling Language Models to 128K Contextcited by 183
  2. 2023BooookScore: A systematic exploration of book-length summarization in the era of LLMscited by 137
  3. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAGcited by 122
  4. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 121
  5. 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 120
  6. 2024LongAlign: A Recipe for Long Context Alignment of Large Language Modelscited by 102
  7. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  8. 2024One Thousand and One Pairs: A "novel" challenge for long-context language modelscited by 80
  9. 2024Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encodingcited by 72
  10. 2022Efficient Long-Text Understanding with Short-Text Modelscited by 71
  11. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?cited by 69
  12. 2024Why Does the Effective Context Length of LLMs Fall Short?cited by 60
  13. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 56
  14. 2024In Defense of RAG in the Era of Long-Context Language Modelscited by 52
  15. 2025LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansioncited by 47
  16. 2024RepoQA: Evaluating Long Context Code Understandingcited by 35
  17. 2024RAR-b: Reasoning as Retrieval Benchmarkcited by 33
  18. 2024Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLPcited by 27
  19. 2024Long Context vs. RAG for LLMs: An Evaluation and Revisitscited by 24
  20. 2024Long Context Alignment with Short Instructions and Synthesized Positionscited by 22
  21. 2024From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Datacited by 18
  22. 2025Eliciting In-context Retrieval and Reasoning for Long-context Large Language Modelscited by 11
  23. 2024Scaling Granite Code Models to 128K Contextcited by 9
  24. 2024A Little Goes a Long Way: Efficient Long Context Training and Inference with Partial Contextscited by 9
  25. 2025Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contextscited by 9
  26. 2024Evaluating Multilingual Long-Context Models for Retrieval and Reasoningcited by 8
  27. 2023Training With "Paraphrasing the Original Text" Teaches LLM to Better Retrieve in Long-context Taskscited by 5
  28. 2025Attention Reveals More Than Tokens: Training-Free Long-Context Reasoning with Attention-guided Retrievalcited by 5
  29. 2025Can LLMs reason over extended multilingual contexts? Towards long-context evaluation beyond retrieval and haystackscited by 3
  30. 2025Reasoning on Multiple Needles In A Haystackcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 110.

118 papers.

Sort:

By area:

  1. 2026SkillReducer: Optimizing LLM Agent Skills for Token Efficiencycited by 29
  2. 2026From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agentscited by 29
  3. 2026Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgetscited by 28
  4. 2026GAM: Hierarchical Graph-based Agentic Memory for LLM Agentscited by 19
  5. 2026APEX-MEM: Agentic Semi-Structured Memory with Temporal Reasoning for Long-Term Conversational AIcited by 16
  6. 2026Evaluating Memory Structure in LLM Agentscited by 12
  7. 2026On Safety Risks in Experience-Driven Self-Evolving Agentscited by 12
  8. 2026δ-mem: Efficient Online Memory for Large Language Modelscited by 12
  9. 2026SWE-MeM: Learning Adaptive Memory Management for Long-Horizon Coding Agentscited by 12
  10. 2026Structurally Aligned Subtask-Level Memory for Software Engineering Agentscited by 11
  11. 2026Coding Agents are Effective Long-Context Processorscited by 11
  12. 2025ContextPilot: Fast Long-Context Inference via Context Reusecited by 8
38 more
  1. 2026SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?cited by 8
  2. 2026Position: Modular Memory is the Key to Continual Learning Agentscited by 7
  3. 2026Gated Differentiable Working Memory for Long-Context Language Modelingcited by 6
  4. 2026QUASAR: A Universal Autonomous System for Atomistic Simulation and a Benchmark of Its Capabilitiescited by 6
  5. 2026EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detectioncited by 6
  6. 2026Improving Coherence and Persistence in Agentic AI for System Optimizationcited by 6
  7. 2026RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provablycited by 6
  8. 2026The Efficiency Frontier: A Unified Framework for Cost-Performance Optimization in LLM Context Managementcited by 6
  9. 2026RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttentioncited by 6
  10. 2026AgentSPEX: An Agent SPecification and EXecution Languagecited by 5
  11. 2026Slipstream: Trajectory-Grounded Compaction Validation for Long-Horizon Agentscited by 5
  12. 2026Intelligence Degradation in Long-Context LLMs: Critical Threshold Determination via Natural Length Distribution Analysiscited by 4
  13. 2026Automating Computational Reproducibility in Social Science: Comparing Prompt-Based and Agent-Based Approachescited by 4
  14. 2026Parallel Context Compaction for Long-Horizon LLM Agent Servingcited by 4
  15. 2025Natural Language Tools: A Natural Language Approach to Tool Calling In Large Language Agentscited by 3
  16. 2026Agent Skill Framework: Perspectives on the Potential of Small to Medium Language Models in Industrial Environmentscited by 3
  17. 2026Ontology-Constrained Neural Reasoning in Enterprise Agentic Systems: A Neurosymbolic Architecture for Domain-Grounded AI Agentscited by 3
  18. 2026Reasoning Shift: How Context Silently Shortens LLM Reasoningcited by 3
  19. 2026Latent Visual Cache for Video Reasoningcited by 3
  20. 2025Fints: Efficient Inference-Time Personalization for LLMs with Fine-Grained Instance-Tailored Steeringcited by 2
  21. 2025Prune-Then-Plan: Step-Level Calibration for Stable Frontier Exploration in Embodied Question Answeringcited by 2
  22. 2025Mistake Notebook Learning: Batch-Clustered Failures for Training-Free Agent Adaptationcited by 2
  23. 2026A Guide to Large Language Models in Modeling and Simulation: From Core Techniques to Critical Challengescited by 2
  24. 2026LoopGuard: Breaking Self-Reinforcing Attention Loops via Dynamic KV Cache Interventioncited by 2
  25. 2026LLMs with in-context learning for Algorithmic Theoretical Physicscited by 2
  26. 2026To Isolate or to Score? Model-Adaptive Assessment for Cost-Efficient Multi-Agent RAGcited by 2
  27. 2025Training-Free Active Learning Framework in Materials Science with Large Language Modelscited by 1
  28. 2026Can large language models interpret unstructured chat data on dynamic group decision-making processes? Evidence on joint destination choicecited by 1
  29. 2026MemCtrl: Using MLLMs as Active Memory Controllers on Embodied Agentscited by 1
  30. 2026LEAD: Breaking the No-Recovery Bottleneck in Long-Horizon Reasoningcited by 1
  31. 2026Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coveragecited by 1
  32. 2026SEAR: Schema-Based Evaluation and Routing for LLM Gatewayscited by 1
  33. 2026View-oriented Conversation Compiler for Agent Trace Analysiscited by 1
  34. 2026Self-Consolidating Language Models: Continual Knowledge Incorporation from Contextcited by 1
  35. 2026Causely: A Causal Intelligence Layer for Enterprise AI A Benchmark Study on SRE and Reliability Workflowscited by 1
  36. 2026Chain-based Adaptive Reconfiguration Over Lattices for Hallucination Reductioncited by 1
  37. 2026How Task Structure Limits Multi-Agent Success: An Information-Theoretic Analysiscited by 1
  38. 2026Diagnosing and Mitigating Context Rot in Long-horizon Searchcited by 1

50 most cited of 118 All 118 (JSON)