# HALoGEN: Fantastic LLM Hallucinations and Where to Find Them

arXiv: https://arxiv.org/abs/2501.08292 · HTML: https://mlsanity.com/paper/2501.08292/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jan 2025.

## Built on by (most cited first, 30 of 34)

- [Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation](https://mlsanity.com/paper/2503.19622/) (2025)
- [MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them](https://mlsanity.com/paper/2507.21017/) (2025)
- [AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agents](https://mlsanity.com/paper/2601.06818/) (2026)
- [Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations](https://mlsanity.com/paper/2504.12691/) (2025)
- [QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generation](https://mlsanity.com/paper/2512.19134/) (2025)
- [Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents](https://mlsanity.com/paper/2605.06635/) (2026)
- [LLM Microscope: What Model Internals Reveal About Answer Correctness and Context Utilization](https://mlsanity.com/paper/2510.04013/) (2025)
- [FACTORY: A Challenging Human-Verified Prompt Set for Long-Form Factuality](https://mlsanity.com/paper/2508.00109/) (2025)
- [A Unified Definition of Hallucination: It's The World Model, Stupid!](https://mlsanity.com/paper/2512.21577/) (2025)
- [Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents](https://mlsanity.com/paper/2604.03173/) (2026)
- [Can Humans Dream of Electric Sheep? Human-Written Samples for Fine-Grained Vision-and-Language Hallucination Benchmarking](https://mlsanity.com/paper/2608.01021/) (2026)
- [Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Models](https://mlsanity.com/paper/2510.06107/) (2025)
- [When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs](https://mlsanity.com/paper/2511.07318/) (2025)
- [A Survey on LLM-based News Recommender Systems](https://mlsanity.com/paper/2502.09797/) (2025)
- [Revisiting the Past: Data Unlearning with Model State History](https://mlsanity.com/paper/2506.20941/) (2025)
- [The Unintended Trade-off of AI Alignment:Balancing Hallucination Mitigation and Safety in LLMs](https://mlsanity.com/paper/2510.07775/) (2025)
- [HaluNet: Learning Hallucination Risk from Internal Signals in LLM Question Answering](https://mlsanity.com/paper/2512.24562/) (2025)
- [HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models](https://mlsanity.com/paper/2605.19341/) (2026)
- [HALLMARK: How to Diagnose LLM Citation Verifiers and When to Deploy Them](https://mlsanity.com/paper/2607.18360/) (2026)
- [Do Multimodal Large Language Models Understand Welding?](https://mlsanity.com/paper/2503.16537/) (2025)
- [Machine Mirages: Defining the Undefined](https://mlsanity.com/paper/2506.13990/) (2025)
- [Sample, Align, Synthesize: Graph-Based Response Synthesis with ConGrs](https://mlsanity.com/paper/2510.03527/) (2025)
- [SEPS: Semantic-enhanced Patch Slimming Framework for fine-grained cross-modal alignment](https://mlsanity.com/paper/2511.01390/) (2025)
- [Auditable Context-Aware HFMD Forecasting with Structured LLM Agents](https://mlsanity.com/paper/2511.23276/) (2025)
- [When Looks Do Not Lie: Discourse Structure Guided In-Context Learning for Faithful Diagram Generation](https://mlsanity.com/paper/2601.20476/) (2026)
- [Multi-Grained Vision-Language Alignment for Domain Generalized Person Re-Identification](https://mlsanity.com/paper/2603.14012/) (2026)
- [An Empirical Analysis of Static Analysis Methods for Detection and Mitigation of Code Library Hallucinations](https://mlsanity.com/paper/2604.07755/) (2026)
- [ReverseEOL: Improving Training-free Text Embeddings via Text Reversal in Decoder-only LLMs](https://mlsanity.com/paper/2606.05858/) (2026)
- [Multimodal LLM-Empowered Re-Ranking for Generalizable Person Re-Identification](https://mlsanity.com/paper/2606.16161/) (2026)
- [Does Finetuning with Scientific Data Increase Hallucinations? A Multi-domain Factuality Evaluation of LLMs](https://mlsanity.com/paper/2606.21359/) (2026)

## Abstract (arXiv)

Despite their impressive ability to generate high-quality and fluent text, generative large language models (LLMs) also produce hallucinations: statements that are misaligned with established world knowledge or provided input context. However, measuring hallucination can be challenging, as having humans verify model generations on-the-fly is both expensive and time-consuming. In this work, we release HALoGEN, a comprehensive hallucination benchmark consisting of: (1) 10,923 prompts for generative models spanning nine domains including programming, scientific attribution, and summarization, and (2) automatic high-precision verifiers for each use case that decompose LLM generations into atomic units, and verify each unit against a high-quality knowledge source. We use this framework to evaluate ~150,000 generations from 14 language models, finding that even the best-performing models are riddled with hallucinations (sometimes up to 86% of generated atomic facts depending on the domain). We further define a novel error classification for LLM hallucinations based on whether they likely stem from incorrect recollection of training data (Type A errors), or incorrect knowledge in training data (Type B errors), or are fabrication (Type C errors). We hope our framework provides a foundation to enable the principled study of why generative models hallucinate, and advances the development of trustworthy large language models.