# Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models

arXiv: https://arxiv.org/abs/2407.04121 · HTML: https://mlsanity.com/paper/2407.04121/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 89 references that are in the map.
- First posted on arXiv 4 Jul 2024.

## Builds on (most cited first, 30 of 60)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization](https://mlsanity.com/paper/2005.03754/) (2020)

## Built on by (most cited first, 30 of 44)

- [A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions](https://mlsanity.com/paper/2412.05563/) (2024)
- [Visual Agents as Fast and Slow Thinkers](https://mlsanity.com/paper/2408.08862/) (2024)
- [Atomic Fact Decomposition Helps Attributed Question Answering](https://mlsanity.com/paper/2410.16708/) (2024)
- [A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs](https://mlsanity.com/paper/2505.08200/) (2025)
- [The Order Effect: Investigating Prompt Sensitivity to Input Order in LLMs](https://mlsanity.com/paper/2502.04134/) (2025)
- [Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perception](https://mlsanity.com/paper/2410.12788/) (2024)
- [Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs](https://mlsanity.com/paper/2510.01218/) (2025)
- [XMeCap: Meme Caption Generation with Sub-Image Adaptability](https://mlsanity.com/paper/2407.17152/) (2024)
- [SINdex: Semantic INconsistency Index for Hallucination Detection in LLMs](https://mlsanity.com/paper/2503.05980/) (2025)
- [Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics](https://mlsanity.com/paper/2505.18658/) (2025)
- [Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Models](https://mlsanity.com/paper/2408.10692/) (2024)
- [DeepNote: Note-Centric Deep Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.08821/) (2024)
- [Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models](https://mlsanity.com/paper/2507.02870/) (2025)
- [Evaluating Evaluation Metrics -- The Mirage of Hallucination Detection](https://mlsanity.com/paper/2504.18114/) (2025)
- [Interpretable LLMs for Credit Risk: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2506.04290/) (2025)
- [Addressing Hallucinations with RAG and NMISS in Italian Healthcare LLM Chatbots](https://mlsanity.com/paper/2412.04235/) (2024)
- [Catch Me if You Search: When Contextual Web Search Results Affect the Detection of Hallucinations](https://mlsanity.com/paper/2504.01153/) (2025)
- [Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings](https://mlsanity.com/paper/2508.06030/) (2025)
- [Tool-MAD: A Multi-Agent Debate Framework for Fact Verification with Diverse Tool Augmentation and Adaptive Retrieval](https://mlsanity.com/paper/2601.04742/) (2026)
- [Can GPT-4 learn to analyse moves in research article abstracts?](https://mlsanity.com/paper/2407.15612/) (2024)
- [Enhancing Exchange Rate Forecasting with Explainable Deep Learning Models](https://mlsanity.com/paper/2410.19241/) (2024)
- [Use Me Wisely: AI-Driven Assessment for LLM Prompting Skills Development](https://mlsanity.com/paper/2503.02532/) (2025)
- [Decomposing and Revising What Language Models Generate](https://mlsanity.com/paper/2509.00765/) (2025)
- [A Geometric Analysis of Small-sized Language Model Hallucinations](https://mlsanity.com/paper/2602.14778/) (2026)
- [Who Checks the Citations? Benchmarking Legal Hallucination Detection](https://mlsanity.com/paper/2606.21155/) (2026)
- [Towards Mitigation of Hallucination for LLM-empowered Agents: Progressive Generalization Bound Exploration and Watchdog Monitor](https://mlsanity.com/paper/2507.15903/) (2025)
- [Bounded PCTL Model Checking of Large Language Model Outputs](https://mlsanity.com/paper/2509.18836/) (2025)
- [Confidence-Aware Routing for Large Language Model Reliability Enhancement: A Multi-Signal Approach to Pre-Generation Hallucination Mitigation](https://mlsanity.com/paper/2510.01237/) (2025)
- [MoM: Mixtures of Scenario-Aware Document Memories for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2510.14252/) (2025)
- [On the Influence of Shape, Texture and Color for Learning Semantic Segmentation](https://mlsanity.com/paper/2410.14878/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) have gained widespread adoption in various natural language processing tasks, including question answering and dialogue systems. However, a major drawback of LLMs is the issue of hallucination, where they generate unfaithful or inconsistent content that deviates from the input source, leading to severe consequences. In this paper, we propose a robust discriminator named RelD to effectively detect hallucination in LLMs' generated answers. RelD is trained on the constructed RelQA, a bilingual question-answering dialogue dataset along with answers generated by LLMs and a comprehensive set of metrics. Our experimental results demonstrate that the proposed RelD successfully detects hallucination in the answers generated by diverse LLMs. Moreover, it performs well in distinguishing hallucination in LLMs' generated answers from both in-distribution and out-of-distribution datasets. Additionally, we also conduct a thorough analysis of the types of hallucinations that occur and present valuable insights. This research significantly contributes to the detection of reliable answers generated by LLMs and holds noteworthy implications for mitigating hallucination in the future work.