mlsanityData to 14 Oct 2026

Are Hallucinations Bad Estimations?

arXiv 2509.21473 ↗First posted 25 Sep 2025cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We formalize hallucinations in generative models as failures to link an estimate to any plausible cause. Under this interpretation, we show that even loss-minimizing optimal estimators still hallucinate. We confirm this with a general high probability lower bound on hallucinate rate for generic data distributions. This reframes hallucination as structural misalignment between loss minimization and human-acceptable outputs, and hence estimation errors induced by miscalibration. Experiments on coin aggregation, open-ended QA, and text-to-image support our theory.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 24 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

19 of 24 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  6. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  7. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  8. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  9. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 357
  10. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
  11. 2025Why Language Models Hallucinatecited by 220
  12. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
7 more
  1. 2023Calibrated Language Models Must Hallucinatecited by 115
  2. 2024Understanding Hallucinations in Diffusion Models through Mode Interpolationcited by 91
  3. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 88
  4. 2024LLMs Will Always Hallucinate, and We Need to Live With Thiscited by 78
  5. 2024A Survey on Large Language Model Hallucination via a Creativity Perspectivecited by 38
  6. 2024Conformalized Credal Regions for Classification with Ambiguous Ground Truthcited by 19
  7. 2017Deriving environmental contours from highest density regionscited by 1

Built on by

Not cited yet.