# The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models

arXiv: https://arxiv.org/abs/2401.03205 · HTML: https://mlsanity.com/paper/2401.03205/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 61 references that are in the map.
- First posted on arXiv 6 Jan 2024.

## Builds on (most cited first, 30 of 39)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [Trusting Your Evidence: Hallucinate Less with Context-aware Decoding](https://mlsanity.com/paper/2305.14739/) (2023)
- [How Language Model Hallucinations Can Snowball](https://mlsanity.com/paper/2305.13534/) (2023)
- [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://mlsanity.com/paper/2209.10063/) (2022)
- [FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://mlsanity.com/paper/2307.13528/) (2023)
- [A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation](https://mlsanity.com/paper/2307.03987/) (2023)
- [Factuality Enhanced Language Models for Open-Ended Text Generation](https://mlsanity.com/paper/2206.04624/) (2022)
- [LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples](https://mlsanity.com/paper/2310.01469/) (2023)
- [Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation](https://mlsanity.com/paper/2307.11019/) (2023)
- [Do Language Models Know When They're Hallucinating References?](https://mlsanity.com/paper/2305.18248/) (2023)
- [The Fact Extraction and VERification (FEVER) Shared Task](https://mlsanity.com/paper/1811.10971/) (2018)
- [Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding](https://mlsanity.com/paper/2104.08455/) (2021)
- [Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answering](https://mlsanity.com/paper/2308.13259/) (2023)
- [Towards Mitigating Hallucination in Large Language Models via Self-Reflection](https://mlsanity.com/paper/2310.06271/) (2023)

## Built on by (most cited first, 30 of 88)

- [Fine-grained Hallucination Detection and Editing for Language Models](https://mlsanity.com/paper/2401.06855/) (2024)
- [HalluLens: LLM Hallucination Benchmark](https://mlsanity.com/paper/2504.17550/) (2025)
- [FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"](https://mlsanity.com/paper/2410.03727/) (2024)
- [PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset](https://mlsanity.com/paper/2403.11116/) (2024)
- [Detecting Hallucinations in Large Language Model Generation: A Token Probability Approach](https://mlsanity.com/paper/2405.19648/) (2024)
- [FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answering](https://mlsanity.com/paper/2405.13873/) (2024)
- [Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning](https://mlsanity.com/paper/2502.02871/) (2025)
- [Content Moderation by LLM: From Accuracy to Legitimacy](https://mlsanity.com/paper/2409.03219/) (2024)
- [HalluDial: A Large-Scale Benchmark for Automatic Dialogue-Level Hallucination Evaluation](https://mlsanity.com/paper/2406.07070/) (2024)
- [DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations](https://mlsanity.com/paper/2410.18860/) (2024)
- [Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Models](https://mlsanity.com/paper/2410.14479/) (2024)
- [OLAPH: Improving Factuality in Biomedical Long-form Question Answering](https://mlsanity.com/paper/2405.12701/) (2024)
- [Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey](https://mlsanity.com/paper/2411.10478/) (2024)
- [Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulness](https://mlsanity.com/paper/2404.00216/) (2024)
- [Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning](https://mlsanity.com/paper/2505.24478/) (2025)
- [Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression](https://mlsanity.com/paper/2505.19433/) (2025)
- [MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them](https://mlsanity.com/paper/2507.21017/) (2025)
- [Refine Knowledge of Large Language Models via Adaptive Contrastive Learning](https://mlsanity.com/paper/2502.07184/) (2025)
- [Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination](https://mlsanity.com/paper/2403.14401/) (2024)
- [Distinguishing Ignorance from Error in LLM Hallucinations](https://mlsanity.com/paper/2410.22071/) (2024)
- [CausalChaos! Dataset for Comprehensive Causal Action Question Answering Over Longer Causal Chains Grounded in Dynamic Visual Scenes](https://mlsanity.com/paper/2404.01299/) (2024)
- [Fakes of Varying Shades: How Warning Affects Human Perception and Engagement Regarding LLM Hallucinations](https://mlsanity.com/paper/2404.03745/) (2024)
- [CoTAR: Chain-of-Thought Attribution Reasoning with Multi-level Granularity](https://mlsanity.com/paper/2404.10513/) (2024)
- [FactCheckmate: Preemptively Detecting and Mitigating Hallucinations in LMs](https://mlsanity.com/paper/2410.02899/) (2024)
- [Uncertainty-Aware Decoding with Minimum Bayes Risk](https://mlsanity.com/paper/2503.05318/) (2025)
- [OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG](https://mlsanity.com/paper/2601.09028/) (2026)
- [SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations](https://mlsanity.com/paper/2510.04398/) (2025)
- [HalluShift: Measuring Distribution Shifts towards Hallucination Detection in LLMs](https://mlsanity.com/paper/2504.09482/) (2025)
- [An Empirical Study on Prompt Compression for Large Language Models](https://mlsanity.com/paper/2505.00019/) (2025)
- [DefAn: Definitive Answer Dataset for LLMs Hallucination Evaluation](https://mlsanity.com/paper/2406.09155/) (2024)

## Abstract (arXiv)

In the era of large language models (LLMs), hallucination (i.e., the tendency to generate factually incorrect content) poses great challenge to trustworthy and reliable deployment of LLMs in real-world applications. To tackle the LLM hallucination, three key questions should be well studied: how to detect hallucinations (detection), why do LLMs hallucinate (source), and what can be done to mitigate them (mitigation). To address these challenges, this work presents a systematic empirical study on LLM hallucination, focused on the the three aspects of hallucination detection, source and mitigation. Specially, we construct a new hallucination benchmark HaluEval 2.0, and designs a simple yet effective detection method for LLM hallucination. Furthermore, we zoom into the different training or utilization stages of LLMs and extensively analyze the potential factors that lead to the LLM hallucination. Finally, we implement and examine a series of widely used techniques to mitigate the hallucinations in LLMs. Our work has led to several important findings to understand the hallucination origin and mitigate the hallucinations in LLMs. Our code and data can be accessed at https://github.com/RUCAIBox/HaluEval-2.0.