Diagnosing Evidence Utilization in Long-Context and Retrieval-Augmented Language Models under Matched Evidence Conditions
On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval
Abstract (arXiv)
Final-answer accuracy, retrieval recall, and citation overlap do not reveal how much answer advantage a long-context or retrieval-augmented language model actually recovers from supplied evidence. A model may answer from parametric priors, fail to use evidence that is present, or cite relevant text without converting it into the final answer. This paper introduces a four-condition diagnostic protocol for evidence-utilization evaluation under matched examples, models, prompts, and scoring rules. The protocol compares no-evidence, full-context, retrieved-evidence, and oracle-evidence reference conditions, and uses Oracle-Reference Normalized Context Utilization (ONCU) as a denominator-valid estimate of recovered oracle-reference evidence advantage. The empirical study evaluates five local open-weight models from the Qwen, Gemma, Llama, and Mistral families over Controlled-ONCU-safe16K, HotpotQA-ONCU, and 2WikiMultiHopQA-ONCU, comprising 18,000 ONCU-compatible predictions. Results show a task-dependent diagnostic pattern: controlled synthetic settings expose reduced recovery when the same evidence is embedded in long input rather than supplied compactly, while realistic multi-hop reconstructions show that full-context inputs outperform the tested retrieved inputs in denominator-free answer and evidence metrics, with ONCU supporting the same direction on oracle-improving groups. Sensitivity audits with stronger retrieval settings narrow some gaps but do not overturn the scoped interpretation. The main contribution is therefore not a single utilization ratio, but a matched diagnostic protocol that separates no-evidence answerability, oracle-evidence recoverability, full-context recovery, retrieval-conditioned recovery, denominator validity, and companion answer/evidence diagnostics.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 8 of its 19 references that are in the map. First posted on arXiv 4 Jun 2026.
Builds on
8 of 19 references.
References · 8
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2025Gemma 3 Technical Reportcited by 1,686
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
- 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2024Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style
- 2026Stale-Document Poisoning: When Outdated Retrieval Overrides Correct Model Answers
- 2026CUE-R: Beyond the Final Answer in Retrieval-Augmented Generation
- 2026Context Shapes LLMs Retrieval-Augmented Fact-Checking Effectiveness
- 2026When Retrieval Helps and Distracts: Evaluating Evidence-Generating LLMs for Biomedical Claim Verification
- 2023Evaluating the Effectiveness of Retrieval-Augmented Large Language Models in Scientific Document Reasoning
- 2026Facet-Level Tracing of Evidence Uncertainty and Hallucination in RAG
- 2024L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding?