# The HalluRAG Dataset: Detecting Closed-Domain Hallucinations in RAG Applications Using an LLM's Internal States

arXiv: https://arxiv.org/abs/2412.17056 · HTML: https://mlsanity.com/paper/2412.17056/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 22 Dec 2024.

## Builds on (most cited first, 19 of 19)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection](https://mlsanity.com/paper/2402.03744/) (2024)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [Do Language Models Know When They're Hallucinating References?](https://mlsanity.com/paper/2305.18248/) (2023)
- [Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering](https://mlsanity.com/paper/2404.17723/) (2024)
- [Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models](https://mlsanity.com/paper/2403.06448/) (2024)
- [Chainpoll: A high efficacy method for LLM hallucination detection](https://mlsanity.com/paper/2310.18344/) (2023)
- [Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference](https://mlsanity.com/paper/2310.03184/) (2023)
- [RefChecker: Reference-based Fine-grained Hallucination Checker and Benchmark for Large Language Models](https://mlsanity.com/paper/2405.14486/) (2024)
- [Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modeling](https://mlsanity.com/paper/2409.11283/) (2024)
- [Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course](https://mlsanity.com/paper/2408.04645/) (2024)

## Built on by (most cited first, 10 of 10)

- [LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signals](https://mlsanity.com/paper/2509.21875/) (2025)
- [Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models](https://mlsanity.com/paper/2507.02870/) (2025)
- [Hallucination Detection: A Probabilistic Framework Using Embeddings Distance Analysis](https://mlsanity.com/paper/2502.08663/) (2025)
- [SEReDeEP: Hallucination Detection in Retrieval-Augmented Models via Semantic Entropy and Context-Parameter Fusion](https://mlsanity.com/paper/2505.07528/) (2025)
- [VeriTrail: Closed-Domain Hallucination Detection with Traceability](https://mlsanity.com/paper/2505.21786/) (2025)
- [Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention](https://mlsanity.com/paper/2602.18145/) (2026)
- [Classifying German Language Proficiency Levels Using Large Language Models](https://mlsanity.com/paper/2512.06483/) (2025)
- [CORTEX: Token-Level Hallucination Detection in RAG via Comparative Internal Representations](https://mlsanity.com/paper/2606.31033/) (2026)
- [HARPO: Hallucination-Aware Reinforcement Learning for Faithful and Creative Language Generation](https://mlsanity.com/paper/2610.03063/) (2026)
- [Grounding Probes: Generator-Independent Hallucination Detection from Observer Model Hidden States](https://mlsanity.com/paper/2610.04642/) (2026)

## Abstract (arXiv)

Detecting hallucinations in large language models (LLMs) is critical for enhancing their reliability and trustworthiness. Most research focuses on hallucinations as deviations from information seen during training. However, the opaque nature of an LLM's parametric knowledge complicates the understanding of why generated texts appear ungrounded: The LLM might not have picked up the necessary knowledge from large and often inaccessible datasets, or the information might have been changed or contradicted during further training. Our focus is on hallucinations involving information not used in training, which we determine by using recency to ensure the information emerged after a cut-off date. This study investigates these hallucinations by detecting them at sentence level using different internal states of various LLMs. We present HalluRAG, a dataset designed to train classifiers on these hallucinations. Depending on the model and quantization, MLPs trained on HalluRAG detect hallucinations with test accuracies ranging up to 75 %, with Mistral-7B-Instruct-v0.1 achieving the highest test accuracies. Our results show that IAVs detect hallucinations as effectively as CEVs and reveal that answerable and unanswerable prompts are encoded differently as separate classifiers for these categories improved accuracy. However, HalluRAG showed some limited generalizability, advocating for more diversity in datasets on hallucinations.