mlsanityData to 7 Oct 2026

Recall Is Not Enough: A Reader-Context Diagnostic for Budget-Constrained Retrieval-Augmented Generation

arXiv 2607.00725 ↗First posted 1 Jul 2026cs.CL cs.IRcited by 1

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-augmented generation under a fixed context budget forces a selection problem: only a fraction of the retrieved evidence fits in front of the reader. The field's standard metric, recall@k, is scored on the retrieved set, but the reader consumes the packed context - and once packing must discard evidence, the two come apart. We introduce answer-in-context, a diagnostic that measures whether a gold answer survives into the packed context, and argue it is the quantity budgeted RAG should be optimizing. It carries substantial information beyond retrieval, adding Delta R^2 = 0.17-0.27 over recall across three multi-hop datasets; even among questions where all gold was retrieved, whether packing keeps the answer separates exact match by 4.6x. Two independent interventions confirm the mediation: a packing change that raises document coverage without raising answer-in-context leaves accuracy flat, and prompt compression that destroys the answer span lowers both together. A graded variant extends the diagnostic to free-form answers, where no verbatim span exists. We then show the diagnostic is actionable. Casting reader-context construction as budgeted submodular maximization gives a packer that beats both deployed top-k truncation and LLMLingua-2 compression - across three reader families, four scales, and four budgets, at equal-or-lower token cost. Against a hand-tuned query-focused heuristic, which we show approximates the same objective, it reaches parity, winning outright only where evidence density is the binding constraint. Throughout, one variable predicts what helps and what cannot.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 33 of its 50 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

33 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  11. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  12. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
20 more
  1. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  2. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  4. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  5. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  6. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  7. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  8. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  9. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  10. 2019Passage Re-ranking with BERTcited by 629
  11. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  12. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  13. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  14. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
  15. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  16. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  17. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  18. 2017Constructing Datasets for Multi-hop Reading Comprehension Across Documentscited by 190
  19. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 160
  20. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 115

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Prompt-Based Abstention Fails Under Misleading Context: A Controlled Study of Small Frozen RAG Models
  2. 2026Separating Semantic Competition from Context Length in RAG Reading
  3. 2025SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression
  4. 2026Know Before You Fetch: Calibrated Retrieval-Budget Allocation for Retrieval-Augmented Generation
  5. 2026Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG
  6. 2026On the impact of retrieved content representations in RAG Pipelines
  7. 2026CUE-R: Beyond the Final Answer in Retrieval-Augmented Generation
  8. 2026Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG