mlsanityData to 14 Oct 2026

Hallucination is Inevitable: An Innate Limitation of Large Language Models

arXiv 2401.11817 ↗First posted 22 Jan 2024cs.CL cs.AI cs.LGcited by 332

Not on the map. Why?

Cited by 44 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Hallucination has been widely recognized to be a significant drawback for large language models (LLMs). There have been many works that attempt to reduce the extent of hallucination. These efforts have mostly been empirical so far, which cannot answer the fundamental question whether it can be completely eliminated. In this paper, we formalize the problem and show that it is impossible to eliminate hallucination in LLMs. Specifically, we define a formal world where hallucination is defined as inconsistencies between a computable LLM and a computable ground truth function. By employing results from learning theory, we show that LLMs cannot learn all the computable functions and will therefore inevitably hallucinate if used as general problem solvers. Since the formal world is a part of the real world which is much more complicated, hallucinations are also inevitable for real world LLMs. Furthermore, for real world LLMs constrained by provable time complexity, we describe the hallucination-prone tasks and empirically validate our claims. Finally, using the formal world framework, we discuss the possible mechanisms and efficacies of existing hallucination mitigators as well as the practical implications on the safe deployment of LLMs.

Key facts. Cited by 332 papers in the map (data through 14 Oct 2026). Builds on 47 of its 72 references that are in the map. First posted on arXiv 22 Jan 2024.

Builds on

47 of 72 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  7. 2022Emergent Abilities of Large Language Modelscited by 2,512
  8. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  10. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  11. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  12. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 803
35 more
  1. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  3. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  4. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  5. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  6. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  7. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
  8. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 341
  9. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 327
  10. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
  11. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  12. 2019Theoretical Limitations of Self-Attention in Neural Sequence Modelscited by 288
  13. 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 260
  14. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  15. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  16. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
  17. 2022Large Language Models with Controllable Working Memorycited by 169
  18. 2017Breaking the Softmax Bottleneck: A High-Rank RNN Language Modelcited by 152
  19. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  20. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 131
  21. 2022Overcoming a Theoretical Limitation of Self-Attentioncited by 113
  22. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  23. 2023SCOTT: Self-Consistent Chain-of-Thought Distillationcited by 105
  24. 2019Assessing The Factual Accuracy of Generated Textcited by 103
  25. 2021The Factual Inconsistency Problem in Abstractive Text Summarization: A Surveycited by 79
  26. 2023Exposing Attention Glitches with Flip-Flop Language Modelingcited by 75
  27. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 61
  28. 2022Entity Cloze By Date: What LMs Know About Unseen Entitiescited by 55
  29. 2023Nationality Bias in Text Generationcited by 50
  30. 2019Embedding Symbolic Knowledge into Deep Networkscited by 49
  31. 2024Hallucination Detection and Hallucination Mitigation: An Investigationcited by 21
  32. 2024Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventionscited by 20
  33. 2021Identifying Untrustworthy Samples: Data Filtering for Open-domain Dialogues with Bayesian Optimizationcited by 15
  34. 2023Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Modelscited by 13
  35. 2024The Pitfalls of Defining Hallucinationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 104; 2025: 157; 2026: 71.

332 papers.

Sort:

By area:

  1. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  2. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  3. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  4. 2024Emergent social conventions and collective bias in LLM populationscited by 123
  5. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  6. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 101
  7. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 99
  8. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 88
  9. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 88
  10. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 77
  11. 2025AgentRxiv: Towards Collaborative Autonomous Researchcited by 72
  12. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Modelscited by 69
38 more
  1. 2024Building Guardrails for Large Language Modelscited by 65
  2. 2025Large language models for automated scholarly paper review: A surveycited by 61
  3. 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?cited by 58
  4. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
  5. 2024Determinants of LLM-assisted Decision-Makingcited by 56
  6. 2024A Closer Look at Machine Unlearning for Large Language Modelscited by 56
  7. 2024Visual Agents as Fast and Slow Thinkerscited by 53
  8. 2024RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learningcited by 49
  9. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
  10. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 46
  11. 2024Language Generation in the Limitcited by 40
  12. 2024Benchmarking Language Model Creativity: A Case Study on Code Generationcited by 40
  13. 2024FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answeringcited by 39
  14. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 38
  15. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 36
  16. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewardscited by 33
  17. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  18. 2024LLaMP: Large Language Model Made Powerful for High-fidelity Materials Knowledge Retrieval and Distillationcited by 32
  19. 2024On the Challenges and Opportunities in Generative AIcited by 31
  20. 2024Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoningcited by 31
  21. 2025Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinkingcited by 31
  22. 2025Prover Agent: An Agent-Based Framework for Formal Mathematical Proofscited by 31
  23. 2024Content Moderation by LLM: From Accuracy to Legitimacycited by 30
  24. 2025AVCD: Mitigating Hallucinations in Audio-Visual Large Language Models through Contrastive Decodingcited by 30
  25. 2024IterGen: Iterative Semantic-aware Structured LLM Generation with Backtrackingcited by 29
  26. 2024Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspectivecited by 29
  27. 2024Exploring Facets of Language Generation in the Limitcited by 28
  28. 2025TruthRL: Incentivizing Truthful LLMs via Reinforcement Learningcited by 27
  29. 2024An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generationcited by 25
  30. 2024LVLM-Interpret: An Interpretability Tool for Large Vision-Language Modelscited by 24
  31. 2025Normative Evaluation of Large Language Models with Everyday Moral Dilemmascited by 24
  32. 2025Hallucination Detection in Large Language Models with Metamorphic Relationscited by 24
  33. 2025Get Experience from Practice: LLM Agents with Record & Replaycited by 23
  34. 2024Testing and Understanding Erroneous Planning in LLM Agents through Synthesized User Inputscited by 22
  35. 2025Density Measures for Language Generationcited by 21
  36. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Modelscited by 21
  37. 2024Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detectorcited by 20
  38. 2024The Fusion of Large Language Models and Formal Methods for Trustworthy AI Agents: A Roadmapcited by 20

50 most cited of 332 All 332 (JSON)