# Why Language Models Hallucinate

arXiv: https://arxiv.org/abs/2509.04664 · HTML: https://mlsanity.com/paper/2509.04664/

## Key facts

- Cited by 220 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 82 references that are in the map.
- First posted on arXiv 4 Sep 2025.

## Builds on (most cited first, 30 of 44)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"](https://mlsanity.com/paper/2309.12288/) (2023)
- [How Language Model Hallucinations Can Snowball](https://mlsanity.com/paper/2305.13534/) (2023)
- [Do Large Language Models Know What They Don't Know?](https://mlsanity.com/paper/2305.18153/) (2023)
- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://mlsanity.com/paper/2405.05904/) (2024)
- [Fine-tuning Language Models for Factuality](https://mlsanity.com/paper/2311.08401/) (2023)
- [WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild](https://mlsanity.com/paper/2406.04770/) (2024)
- [Reducing conversational agents' overconfidence through linguistic calibration](https://mlsanity.com/paper/2012.14983/) (2020)
- [Factuality Enhanced Language Models for Open-Ended Text Generation](https://mlsanity.com/paper/2206.04624/) (2022)
- [MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning](https://mlsanity.com/paper/2310.16049/) (2023)
- [Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?](https://mlsanity.com/paper/2308.10168/) (2023)
- [Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty](https://mlsanity.com/paper/2507.16806/) (2025)
- [Do Language Models Know When They're Hallucinating References?](https://mlsanity.com/paper/2305.18248/) (2023)
- [Calibrated Language Models Must Hallucinate](https://mlsanity.com/paper/2311.14648/) (2023)

## Built on by (most cited first, 30 of 220)

- [TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning](https://mlsanity.com/paper/2509.25760/) (2025)
- [From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2509.25373/) (2025)
- [Training LLMs for Honesty via Confessions](https://mlsanity.com/paper/2512.08093/) (2025)
- [Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark](https://mlsanity.com/paper/2509.26574/) (2025)
- [Spherical Steering: Geometry-Aware Activation Rotation for Language Models](https://mlsanity.com/paper/2602.08169/) (2026)
- [Trace Length is a Simple Uncertainty Signal in Reasoning Models](https://mlsanity.com/paper/2510.10409/) (2025)
- [Mitigating LLM Hallucination via Behaviorally Calibrated Reinforcement Learning](https://mlsanity.com/paper/2512.19920/) (2025)
- [RAG Meets Temporal Graphs: Time-Sensitive Modeling and Retrieval for Evolving Knowledge](https://mlsanity.com/paper/2510.13590/) (2025)
- [Over-Searching in Search-Augmented Large Language Models](https://mlsanity.com/paper/2601.05503/) (2026)
- [Evaluating LLMs in Finance Requires Explicit Bias Consideration](https://mlsanity.com/paper/2602.14233/) (2026)
- [Failure Modes in LLM Systems: A System-Level Taxonomy for Reliable AI Applications](https://mlsanity.com/paper/2511.19933/) (2025)
- [Causal Evidence that Language Models use Confidence to Drive Behavior](https://mlsanity.com/paper/2603.22161/) (2026)
- [Language Generation: Complexity Barriers and Implications for Learning](https://mlsanity.com/paper/2511.05759/) (2025)
- [Hallucination Begins Where Saliency Drops](https://mlsanity.com/paper/2601.20279/) (2026)
- [The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillation](https://mlsanity.com/paper/2604.16830/) (2026)
- [MedMMV: A Controllable Multimodal Multi-Agent Framework for Reliable and Verifiable Clinical Reasoning](https://mlsanity.com/paper/2509.24314/) (2025)
- [ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spaces](https://mlsanity.com/paper/2602.11683/) (2026)
- [Mixture-of-Minds: Multi-Agent Reinforcement Learning for Table Understanding](https://mlsanity.com/paper/2510.20176/) (2025)
- [FAPO: Flawed-Aware Policy Optimization for Efficient and Reliable Reasoning](https://mlsanity.com/paper/2510.22543/) (2025)
- [TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework](https://mlsanity.com/paper/2511.05385/) (2025)
- [When Do LLM Preferences Predict Downstream Behavior?](https://mlsanity.com/paper/2602.18971/) (2026)
- [From Conflict to Consensus: Boosting Medical Reasoning via Multi-Round Agentic RAG](https://mlsanity.com/paper/2603.03292/) (2026)
- [BAS: A Decision-Theoretic Approach to Evaluating Large Language Model Confidence](https://mlsanity.com/paper/2604.03216/) (2026)
- [BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation](https://mlsanity.com/paper/2606.30850/) (2026)
- [PsychiatryBench: A Multi-Task Benchmark for LLMs in Psychiatry](https://mlsanity.com/paper/2509.09711/) (2025)
- [Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learning](https://mlsanity.com/paper/2510.02324/) (2025)
- [AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning](https://mlsanity.com/paper/2512.16883/) (2025)
- [Reaching Beyond the Mode: RL for Distributional Reasoning in Language Models](https://mlsanity.com/paper/2603.24844/) (2026)
- [ChartHal: A Fine-grained Framework Evaluating Hallucination of Large Vision Language Models in Chart Understanding](https://mlsanity.com/paper/2509.17481/) (2025)
- [Cognitive Load Limits in Large Language Models: Benchmarking Multi-Hop Reasoning](https://mlsanity.com/paper/2509.19517/) (2025)

## Abstract (arXiv)

Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty. Such "hallucinations" persist even in state-of-the-art systems and undermine trust. We argue that language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty, and we analyze the statistical causes of hallucinations in the modern training pipeline. Hallucinations need not be mysterious -- they originate simply as errors in binary classification. If incorrect statements cannot be distinguished from facts, then hallucinations in pretrained language models will arise through natural statistical pressures. We then argue that hallucinations persist due to the way most evaluations are graded -- language models are optimized to be good test-takers, and guessing when uncertain improves test performance. This "epidemic" of penalizing uncertain responses can only be addressed through a socio-technical mitigation: modifying the scoring of existing benchmarks that are misaligned but dominate leaderboards, rather than introducing additional hallucination evaluations. This change may steer the field toward more trustworthy AI systems.