mlsanityData to 7 Oct 2026

The Semantic Illusion: Certified Limits of Embedding-Based Hallucination Detection in RAG Systems

arXiv 2512.15068 ↗First posted 17 Dec 2025cs.LG cs.AI cs.CLcited by 0

On the map: Retrieval-augmented generation → Hallucination detection in RAG

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) systems remain susceptible to hallucinations despite grounding in retrieved evidence. While current detection methods leverage embedding similarity and natural language inference (NLI), their reliability in safety-critical settings remains unproven. We apply conformal prediction to RAG hallucination detection, transforming heuristic scores into decision sets with finite-sample coverage guarantees (1-alpha). Using calibration sets of n=600, we demonstrate a fundamental dichotomy: on synthetic hallucinations (Natural Questions), embedding methods achieve 95% coverage with 0% False Positive Rate (FPR). However, on real hallucinations from RLHF-aligned models (HaluEval), the same methods fail catastrophically, yielding 100% FPR at target coverage. We analyze this failure through the lens of distributional tails, showing that while NLI models achieve acceptable AUC (0.81), the "hardest" hallucinations are semantically indistinguishable from faithful responses, forcing conformal thresholds to reject nearly all valid outputs. Crucially, GPT-4 as a judge achieves 7% FPR (95% CI:[3.4%, 13.7%]) on the same data, proving the task is solvable via reasoning but opaque to surface-level semantics--a phenomenon we term the "Semantic Illusion."

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation. Builds on 44 of its 59 references that are in the map. First posted on arXiv 17 Dec 2025.

Builds on

44 of 59 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignment9 Dec
  2. 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability15 Oct
  3. 2024LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagation28 Aug
  4. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models31 Dec

Other references · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  9. 2017On Calibration of Modern Neural Networkscited by 4,860
  10. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  11. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  12. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
28 more
  1. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  2. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  4. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  8. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  9. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  10. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  11. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  12. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  13. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  14. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  15. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  16. 2019Conformalized Quantile Regressioncited by 620
  17. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  18. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  19. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  20. 2020Selective Question Answering under Domain Shiftcited by 228
  21. 2019Predictive inference with the jackknife+cited by 206
  22. 2023Conformal Language Modelingcited by 163
  23. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 131
  24. 2024Mitigating LLM Hallucinations via Conformal Abstentioncited by 56
  25. 2023Conformal Nucleus Samplingcited by 32
  26. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  27. 2025Hallucination Detection: A Probabilistic Framework Using Embeddings Distance Analysiscited by 5
  28. 2025HALT-RAG: A Task-Adaptable Framework for Hallucination Detection with Calibrated NLI Ensembles and Abstentioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration
  2. 2025LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signals
  3. 2025Probabilistic distances-based hallucination detection in LLMs with RAG
  4. 2025InterpDetect: Interpretable Signals for Detecting Hallucinations in Retrieval-Augmented Generation
  5. 2025Osiris: A Lightweight Open-Source Hallucination Detection System
  6. 2026Detecting Hallucinations in Retrieval-Augmented Generation via Semantic-level Internal Reasoning Graph
  7. 2025Revisiting Hallucination Detection with Effective Rank-based Uncertainty
  8. 2025TPA: Next Token Probability Attribution for Detecting Hallucinations in RAG