mlsanityData to 14 Oct 2026

Sufficient Context: A New Lens on Retrieval Augmented Generation Systems

arXiv 2411.06037 ↗First posted 9 Nov 2024cs.CLcited by 72

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Augmenting LLMs with context leads to improved performance across many applications. Despite much research on Retrieval Augmented Generation (RAG) systems, an open question is whether errors arise because LLMs fail to utilize the context from retrieval or the context itself is insufficient to answer the query. To shed light on this, we develop a new notion of sufficient context, along with a method to classify instances that have enough information to answer the query. We then use sufficient context to analyze several models and datasets. By stratifying errors based on context sufficiency, we find that larger models with higher baseline performance (Gemini 1.5 Pro, GPT 4o, Claude 3.5) excel at answering queries when the context is sufficient, but often output incorrect answers instead of abstaining when the context is not. On the other hand, smaller models with lower baseline performance (Mistral 3, Gemma 2) hallucinate or abstain often, even with sufficient context. We further categorize cases when the context is useful, and improves accuracy, even though it does not fully answer the query and the model errs without the context. Building on our findings, we explore ways to reduce hallucinations in RAG systems, including a new selective generation method that leverages sufficient context information for guided abstention. Our method improves the fraction of correct answers among times where the model responds by 2--10% for Gemini, GPT, and Gemma. Key findings and the prompts used in our autorater analysis are available on our github.

Key facts. Cited by 72 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 43 of its 52 references that are in the map. First posted on arXiv 9 Nov 2024.

Builds on

43 of 52 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation19 Sep
  2. 2024In Defense of RAG in the Era of Long-Context Language Models3 Sep
  3. 2024REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering27 Feb
  4. 2024Metacognitive Retrieval-Augmented Large Language Models18 Feb
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 38

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  8. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  9. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  10. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  11. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  12. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
26 more
  1. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
  2. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  3. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  4. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
  5. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
  6. 2021Measuring Attribution in Natural Language Generation Modelscited by 256
  7. 2023Do Large Language Models Know What They Don't Know?cited by 243
  8. 2024Corrective Retrieval Augmented Generationcited by 197
  9. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  10. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 166
  11. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 149
  12. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 142
  13. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 111
  14. 2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 101
  15. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 77
  16. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 74
  17. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  18. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 61
  19. 2024How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?cited by 44
  20. 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
  21. 2023End-to-End Beam Retrieval for Multi-Hop Question Answeringcited by 35
  22. 2024BERGEN: A Benchmarking Library for Retrieval-Augmented Generationcited by 31
  23. 2024Constructing Benchmarks and Interventions for Combating Hallucinations in LLMscited by 19
  24. 2023Participatory Personalization in Classificationcited by 5
  25. 2024Classification with Conceptual Safeguardscited by 4
  26. 2024Online Joint Fine-tuning of Multi-Agent Flowscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 22; 2026: 49.

72 papers.

Sort:

By area:

  1. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 99
  2. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontierscited by 27
  3. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignmentcited by 23
  4. 2025Controlled Retrieval-augmented Context Evaluation for Long-form RAGcited by 9
  5. 2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
  6. 2025Uncertainty Quantification in Retrieval Augmented Question Answeringcited by 8
  7. 2025Memory in Large Language Models: Mechanisms, Evaluation and Evolutioncited by 8
  8. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surfacecited by 8
  9. 2026Know When You're Wrong: Aligning Confidence with Correctness for LLM Error Detectioncited by 8
  10. 2025When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorationscited by 7
  11. 2025Influence Guided Context Selection for Effective Retrieval-Augmented Generationcited by 7
  12. 2025Towards Geo-Culturally Grounded LLM Generationscited by 5
38 more
  1. 2025FIRESPARQL: A LLM-based Framework for SPARQL Query Generation over Scholarly Knowledge Graphscited by 5
  2. 2025Prompting Large Language Models with Partial Knowledge for Answering Questions with Unseen Entitiescited by 4
  3. 2025Knowing What's Missing: Assessing Information Sufficiency in Question Answeringcited by 3
  4. 2026BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Searchcited by 3
  5. 2026What Context Does a Coding Agent Actually Need to Act?cited by 3
  6. 2026HALT: Verification-Aware Stopping for Retrieval-Augmented Search Agentscited by 3
  7. 2025Copy-Paste to Mitigate Large Language Model Hallucinationscited by 2
  8. 2025Knowledge-Grounded Agentic Large Language Models for Multi-Hazard Understanding from Reconnaissance Reportscited by 2
  9. 2026How Retrieved Context Shapes Internal Representations in RAGcited by 2
  10. 2026Agentic Multi-Source Grounding for Enhanced Query Intent Understanding: A DoorDash Case Studycited by 2
  11. 2026Context-Agent: Dynamic Discourse Trees for Non-Linear Dialoguecited by 2
  12. 2026Cartridges++: KV Cache Compression without Off-Context Derailmentcited by 2
  13. 2025AdvisingWise: Supporting Academic Advising in Higher Education Settings Through a Human-in-the-Loop Multi-Agent Frameworkcited by 1
  14. 2025AnimAgents: Coordinating Multi-Stage Animation Pre-Production with Human-Multi-Agent Collaborationcited by 1
  15. 2026Beyond Case Law: Evaluating Structure-Aware Retrieval and Safety in Statute-Centric Legal QAcited by 1
  16. 2026RAG-Enhanced Large Language Models for Dynamic Content Expiration Prediction in Web Searchcited by 1
  17. 2026Evidence Absence Is Not Evidence Insufficiency: Diagnosing NEI Construction Artifacts in Fact Verificationcited by 1
  18. 2026On the impact of retrieved content representations in RAG Pipelinescited by 1
  19. 2026Discourse-Role Labels as Presentation-Time Variables for Context Use in Language Modelscited by 1
  20. 2026Document-Authored Control-Signal Impersonation: A Low-Cost Indirect Prompt Attack on RAG Safety Boundariescited by 1
  21. 2026Task-Aware Harness Provisioning for LLM Agents in Mission-Critical Infrastructure Operationscited by 1
  22. 2026Prompt-Based Abstention Fails Under Misleading Context: A Controlled Study of Small Frozen RAG Modelscited by 1
  23. 2026Twin Worlds: Equivariance-Based Abstention for Evidence-Grounded Reasoningcited by 1
  24. 2026CoVeR: Coverage-Based Routing of Verifier Calls in Agentic Retrievalcited by 1
  25. 2025MHA-RAG: Improving Efficiency, Accuracy, and Consistency by Encoding Exemplars as Soft Promptscited by 0
  26. 2025Prior Knowledge Makes It Possible: From Sublinear Graph Algorithms to LLM Test-Time Methodscited by 0
  27. 2025Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applicationscited by 0
  28. 2025Retrieval Augmented Generation-Enhanced Distributed LLM Agents for Generalizable Traffic Signal Control with Emergency Vehiclescited by 0
  29. 2025Needle in the Web: A Benchmark for Retrieving Targeted Web Pages in the Wildcited by 0
  30. 2026LADFA: A Framework of Using Large Language Models and Retrieval-Augmented Generation for Personal Data Flow Analysis in Privacy Policiescited by 0
  31. 2026When Does Context Help? Error Dynamics of Contextual Information in Large Language Modelscited by 0
  32. 2026Hypothesis-Conditioned Query Rewriting for Decision-Useful Retrievalcited by 0
  33. 2026UnWeaving the knots of GraphRAG -- turns out VectorRAG is almost enoughcited by 0
  34. 2026Reliable AI Needs to Externalize Implicit Knowledge: A Human-AI Collaboration Perspectivecited by 0
  35. 2026Uncertainty Quantification for Multimodal Retrieval Augmented Generationcited by 0
  36. 2026When Retrieval Metrics Mislead: Measuring Policy Signal in Long-Horizon Tool-Use Agentscited by 0
  37. 2026Dual-Confidence Contrastive Decoding for Retrieval-Augmented Generationcited by 0
  38. 2026TokenMem: Faithful Knowledge Injection for Frozen LLMscited by 0

50 most cited of 72 All 72 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
  2. 2024Context Awareness Gate For Retrieval Augmented Generation
  3. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely
  4. 2024Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation
  5. 2025On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems
  6. 2025Dynamic Context Selection for Retrieval-Augmented Generation: Mitigating Distractors and Positional Bias
  7. 2026Can Small Language Models Use What They Retrieve? An Empirical Study of Retrieval Utilization Across Model Scale
  8. 2025Enhancing Retrieval-Augmented Generation: A Study of Best Practices