Recall Is Not Enough: A Reader-Context Diagnostic for Budget-Constrained Retrieval-Augmented Generation
On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval
Abstract (arXiv)
Retrieval-augmented generation under a fixed context budget forces a selection problem: only a fraction of the retrieved evidence fits in front of the reader. The field's standard metric, recall@k, is scored on the retrieved set, but the reader consumes the packed context - and once packing must discard evidence, the two come apart. We introduce answer-in-context, a diagnostic that measures whether a gold answer survives into the packed context, and argue it is the quantity budgeted RAG should be optimizing. It carries substantial information beyond retrieval, adding Delta R^2 = 0.17-0.27 over recall across three multi-hop datasets; even among questions where all gold was retrieved, whether packing keeps the answer separates exact match by 4.6x. Two independent interventions confirm the mediation: a packing change that raises document coverage without raising answer-in-context leaves accuracy flat, and prompt compression that destroys the answer span lowers both together. A graded variant extends the diagnostic to free-form answers, where no verbatim span exists. We then show the diagnostic is actionable. Casting reader-context construction as budgeted submodular maximization gives a packer that beats both deployed top-k truncation and LLMLingua-2 compression - across three reader families, four scales, and four budgets, at equal-or-lower token cost. Against a hand-tuned query-focused heuristic, which we show approximates the same objective, it reaches parity, winning outright only where evidence density is the binding constraint. Throughout, one variable predicts what helps and what cannot.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 33 of its 50 references that are in the map. First posted on arXiv 1 Jul 2026.
Builds on
33 of 50 references.
Closest ancestors · 1
Same direction, newest first
Other references · 32
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
20 more
- 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
- 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
- 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
- 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
- 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
- 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
- 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
- 2023In-Context Retrieval-Augmented Language Modelscited by 694
- 2019Passage Re-ranking with BERTcited by 629
- 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
- 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
- 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
- 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
- 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
- 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
- 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
- 2017Constructing Datasets for Multi-hop Reading Comprehension Across Documentscited by 190
- 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 160
- 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 115
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Prompt-Based Abstention Fails Under Misleading Context: A Controlled Study of Small Frozen RAG Models
- 2026Separating Semantic Competition from Context Length in RAG Reading
- 2025SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression
- 2026Know Before You Fetch: Calibrated Retrieval-Budget Allocation for Retrieval-Augmented Generation
- 2026Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG
- 2026On the impact of retrieved content representations in RAG Pipelines
- 2026CUE-R: Beyond the Final Answer in Retrieval-Augmented Generation
- 2026Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG