SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation
On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval
Abstract (arXiv)
Retrieval-augmented generation (RAG) grounds answers in retrieved passages, yet relevance does not guarantee sufficiency: a topical passage may still fail to justify the answer. We study evidence sufficiency verification for selective RAG answering, in which a verifier receives a question, a candidate answer, and retrieved evidence and decides whether the evidence supports, refutes, or is insufficient for the answer, answering only when support is established. We present SURE-RAG, an aggregation protocol that treats evidence sufficiency as a set-level property: missing hops and unresolved conflicts cannot be detected by scoring passages independently. A shared claim-evidence verifier produces a local relation distribution for each (claim, passage) pair, which SURE-RAG aggregates into four interpretable answer-level feature blocks (coverage, relation strength, uncertainty, and retrieval), producing a three-way decision and an auditable selective score. We evaluate on HotpotQA-RAG v3, a controlled multi-hop benchmark, under an artifact-aware protocol (shortcut baselines, counterfactual swaps, no-oracle checks, and GPT-4o audits). Calibrated SURE-RAG attains 0.9075 Macro-F1 (raw 0.8951 +/- 0.0069), well above DeBERTa mean-pooling (0.6516) and a GPT-4o judge (0.7284), and on par with a strong concat cross-encoder (0.8888 +/- 0.0109) while remaining fully auditable. At 30% coverage, risk falls from 0.2588 to 0.1642, a 37% relative reduction. As a boundary-mapping experiment, we contrast SURE-RAG with GPT-4o on HaluBench unsafe detection: the ranking reverses (0.3343 vs. 0.7389 unsafe-F1), indicating that controlled sufficiency verification and natural hallucination detection are distinct problems.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 20 of its 26 references that are in the map. First posted on arXiv 5 May 2026.
Builds on
20 of 26 references.
References · 20
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2024GPT-4o System Cardcited by 4,467
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
- 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
- 2017Selective Classification for Deep Neural Networkscited by 799
- 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
- 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
8 more
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2020Fact or Fiction: Verifying Scientific Claimscited by 546
- 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
- 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
- 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
- 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
- 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 166
- 2024RORA: Robust Free-Text Rationale Evaluationcited by 4
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Evidence Absence Is Not Evidence Insufficiency: Diagnosing NEI Construction Artifacts in Fact Verificationcited by 1
- 2026HopRefusalBench: Diagnosing Refusal Failures in Search-Augmented Agents for Multi-Hop Reasoningcited by 1
- 2026Evidence-State Reliability Under Controlled Degradation: Parser-Validity Divergence in a Multi-Stage LLM Pipelinecited by 0
- 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Studycited by 0
- 2026Before Answering: Evidence Sufficiency under Size-Matched Memory Constructioncited by 0
- 2026Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAGcited by 0
- 2026Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG27 Sep 2026
- 2026Before Answering: Evidence Sufficiency under Size-Matched Memory Construction26 Sep 2026
- 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study7 Sep 2026
- 2026Evidence-State Reliability Under Controlled Degradation: Parser-Validity Divergence in a Multi-Stage LLM Pipeline21 Aug 2026
- 2026HopRefusalBench: Diagnosing Refusal Failures in Search-Augmented Agents for Multi-Hop Reasoning2 Aug 2026
- 2026Evidence Absence Is Not Evidence Insufficiency: Diagnosing NEI Construction Artifacts in Fact Verification26 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Stateful Evidence-Driven Retrieval-Augmented Generation with Iterative Reasoning
- 2026When Iterative RAG Beats Ideal Evidence: A Diagnostic Study in Scientific Multi-hop Question Answering
- 2026AdaGATE: Adaptive Gap-Aware Token-Efficient Evidence Assembly for Multi-Hop Retrieval-Augmented Generation
- 2026S2G-RAG: Structured Sufficiency and Gap Judging for Iterative Retrieval-Augmented QA
- 2025EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes
- 2026CQC-RAG: Robust Retrieval-Augmented Generation via Cross-Query Consistency
- 2026MEGRAG: Multi-Granular Evidence Graphs for Answer-Aware Multi-Hop RAG
- 2026Rethinking the Reranker: Boundary-Aware Evidence Selection for Robust Retrieval-Augmented Generation