RAGScope: A Leakage-Controlled, Cost-Aware Evidence-Gating Protocol for RAG Hallucination Triage
On the map: Retrieval-augmented generation → Hallucination detection in RAG also fits: Privacy and security of RAG
Abstract (arXiv)
Retrieval-augmented generation (RAG) systems need inexpensive ways to route generated answers: accept low-risk outputs, review uncertain ones, and reserve strong verifiers for the expensive tail. We present RAGScope, a leakage-controlled protocol for evaluating local evidence gates that use only the task input, retrieved context, and answer text. The protocol combines context-grouped splits, fold-scoped preprocessing, group bootstrap intervals, deployment operating points, end-to-end runtime, and explicit source-shift stress tests. On three RAGTruth tasks, the enhanced gate RAGScope-E reaches 0.798 AUROC and 0.660 average precision (AP) in pooled grouped cross-validation. Its pooled AP exceeds ROUGE-L by 0.034 with a 95% context-group interval of [0.002, 0.064], although the AUROC gain is not significant and ROUGE-L remains stronger on data-to-text. At a top-10% review budget, RAGScope-E attains 0.748 precision; accepting the lowest-risk 50% yields 0.141 residual unfaithfulness. RAGScope-E runs in 6.22 ms/example on CPU, versus 145.75 and 223.07 ms/example for the tested DeBERTa-NLI and HHEM settings. A 14,900-example HaluBench stress test exposes the deployment boundary: an in-domain calibrated gate reaches 0.879 AUROC, but leave-source-out calibration averages only 0.466. Target-only calibration recovers to 0.675 AUROC with 100 labels per source and 0.685 with 200. Cheap evidence gates are therefore useful routing components, but learned calibration must be validated and adapted within the target domain.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation. Builds on 11 of its 13 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
11 of 13 references.
Closest ancestors · 2
Same direction, newest first
- 2024Lynx: An Open Source Hallucination Evaluation Model11 Jul
- 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models31 Dec
Other references · 9
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
- 2017Selective Classification for Deep Neural Networkscited by 852
- 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
- 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
- 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 418
- 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 407
- 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 341
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025FVA-RAG: Falsification-Verification Alignment for Mitigating Sycophantic Hallucinations
- 2025LettuceDetect: A Hallucination Detection Framework for RAG Applications
- 2026Why RAGs Hallucinate: Penalty-Aware Evaluation of Retrieval-Augmented Generation Systems with Knowledge-Gap Canaries
- 2025MetaRAG: Metamorphic Testing for Hallucination Detection in RAG Systems
- 2026Retromorphic Testing with Hierarchical Verification for Hallucination Detection in RAG
- 2026Facet-Level Tracing of Evidence Uncertainty and Hallucination in RAG
- 2026What Would Fix This RAG Failure? Auditing Counterfactual Response with Paired Evidence Interventions
- 2026RAGStress: A controlled benchmark for evaluating retrieval-augmented generation under knowledge-base degradation