On the Fundamental Impossibility of Hallucination Control in Large Language Models
Not on the map. Why?
Abstract (arXiv)
Large language models hallucinate. This paper shows when that is unavoidable and what we can do about it. We model inference as an auction of ideas, in which a model's components, each holding partial knowledge, compete to shape the answer. We then prove Impossibility Theorems showing that whenever a query makes LLM components contest a fact they hold in common, no aggregation of their reports can at once report that knowledge truthfully, avoid manufacturing confidence beyond what it supports, keep the relevant components engaged, and give the best answer. Something must give, and each failure is familiar: a fabricated detail, unearned confidence, ignored knowledge, or a needlessly weak reply. This is no artifact of one design. It reappears when components report probabilities, and inside the transformer itself, where the combined answer is credited more confidence than the internal contributions supplied. The unbalanced semantic budget cannot be settled from within. Factual truth lies outside the model, and in the worst case no internal signal can certify it. What can be certified is support. Given externally authorized evidence, checking that an answer stays within what the evidence entails needs only the answer and the evidence, and we prove when that check is computable. However, a correct answer can lack support, and a supported answer can be false. What counts as evidence, how far beyond it we allow answers to reach, and which failures we can live with are choices no model can make for us.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 22 of its 72 references that are in the map. First posted on arXiv 4 Jun 2025.
Builds on
22 of 72 references.
References · 22
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
- 2022Toy Models of Superpositioncited by 971
- 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
- 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 341
- 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
- 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 269
10 more
- 2025Open Problems in Mechanistic Interpretabilitycited by 179
- 2022Faithful Reasoning Using Large Language Modelscited by 130
- 2023Calibrated Language Models Must Hallucinatecited by 115
- 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretabilitycited by 101
- 2024Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answeringcited by 96
- 2024Estimating Knowledge in Large Language Models Without Generating a Single Tokencited by 40
- 2024Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinationscited by 38
- 2025A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputscited by 21
- 2025Hallucinations are inevitable but can be made statistically negligiblecited by 5
- 2024No Free Lunch: Fundamental Limits of Learning Non-Hallucinating Generative Modelscited by 2
Built on by
Citing papers per year: 2025: 3; 2026: 4.
7 papers.
- 2025Robust AI Security and Alignment: A Sisyphean Endeavor?cited by 3
- 2026Hallucination is a Consequence of Space-Optimality: A Rate-Distortion Theorem for Membership Testingcited by 2
- 2025Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisionscited by 1
- 2026Spilled Energy in Large Language Modelscited by 1
- 2025Tractable Asymmetric Verification for Large Language Models via Deterministic Replicabilitycited by 0
- 2026UnWeaving the knots of GraphRAG -- turns out VectorRAG is almost enoughcited by 0
- 2026The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systemscited by 0
- 2026The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systems21 May 2026
- 2026Spilled Energy in Large Language Models21 Feb 2026
- 2026UnWeaving the knots of GraphRAG -- turns out VectorRAG is almost enough6 Feb 2026
- 2026Hallucination is a Consequence of Space-Optimality: A Rate-Distortion Theorem for Membership Testing31 Jan 2026
- 2025Robust AI Security and Alignment: A Sisyphean Endeavor?10 Dec 2025
- 2025Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisions10 Nov 2025
- 2025Tractable Asymmetric Verification for Large Language Models via Deterministic Replicability14 Sep 2025