# On the Fundamental Impossibility of Hallucination Control in Large Language Models

arXiv: https://arxiv.org/abs/2506.06382 · HTML: https://mlsanity.com/paper/2506.06382/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 72 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 22 of 22)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2305.11747/) (2023)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](https://mlsanity.com/paper/2401.01313/) (2024)
- [Open Problems in Mechanistic Interpretability](https://mlsanity.com/paper/2501.16496/) (2025)
- [Faithful Reasoning Using Large Language Models](https://mlsanity.com/paper/2208.14271/) (2022)
- [Calibrated Language Models Must Hallucinate](https://mlsanity.com/paper/2311.14648/) (2023)
- [ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability](https://mlsanity.com/paper/2410.11414/) (2024)
- [Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering](https://mlsanity.com/paper/2404.17723/) (2024)
- [Estimating Knowledge in Large Language Models Without Generating a Single Token](https://mlsanity.com/paper/2406.12673/) (2024)
- [Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations](https://mlsanity.com/paper/2403.18167/) (2024)
- [A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs](https://mlsanity.com/paper/2505.08200/) (2025)
- [Hallucinations are inevitable but can be made statistically negligible](https://mlsanity.com/paper/2502.12187/) (2025)
- [No Free Lunch: Fundamental Limits of Learning Non-Hallucinating Generative Models](https://mlsanity.com/paper/2410.19217/) (2024)

## Built on by (most cited first, 7 of 7)

- [Robust AI Security and Alignment: A Sisyphean Endeavor?](https://mlsanity.com/paper/2512.10100/) (2025)
- [Hallucination is a Consequence of Space-Optimality: A Rate-Distortion Theorem for Membership Testing](https://mlsanity.com/paper/2602.00906/) (2026)
- [Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisions](https://mlsanity.com/paper/2511.07669/) (2025)
- [Spilled Energy in Large Language Models](https://mlsanity.com/paper/2602.18671/) (2026)
- [Tractable Asymmetric Verification for Large Language Models via Deterministic Replicability](https://mlsanity.com/paper/2509.11068/) (2025)
- [UnWeaving the knots of GraphRAG -- turns out VectorRAG is almost enough](https://mlsanity.com/paper/2603.29875/) (2026)
- [The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systems](https://mlsanity.com/paper/2605.23024/) (2026)

## Abstract (arXiv)

Large language models hallucinate. This paper shows when that is unavoidable and what we can do about it. We model inference as an auction of ideas, in which a model's components, each holding partial knowledge, compete to shape the answer. We then prove Impossibility Theorems showing that whenever a query makes LLM components contest a fact they hold in common, no aggregation of their reports can at once report that knowledge truthfully, avoid manufacturing confidence beyond what it supports, keep the relevant components engaged, and give the best answer. Something must give, and each failure is familiar: a fabricated detail, unearned confidence, ignored knowledge, or a needlessly weak reply. This is no artifact of one design. It reappears when components report probabilities, and inside the transformer itself, where the combined answer is credited more confidence than the internal contributions supplied. The unbalanced semantic budget cannot be settled from within. Factual truth lies outside the model, and in the worst case no internal signal can certify it. What can be certified is support. Given externally authorized evidence, checking that an answer stays within what the evidence entails needs only the answer and the evidence, and we prove when that check is computable. However, a correct answer can lack support, and a supported answer can be false. What counts as evidence, how far beyond it we allow answers to reach, and which failures we can live with are choices no model can make for us.