mlsanityData to 7 Oct 2026

Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer

arXiv 2605.22007 ↗First posted 21 May 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Hallucination is often viewed as a direct consequence of missing knowledge: a model answers incorrectly when the correct answer is absent from its generation-time distribution, and correctly when it is present. We test this assumption by introducing a semantic notion of answer availability that aggregates token-level variants expressing the same answer concept, and asks whether the correct concept is already available at the moment the model commits to an answer. Across Qwen and Llama models from 0.8B to 72B in both Instruct and Base variants, 16-47% of Instruct hallucinations occur with substantial probability mass already on the correct concept, and the rate rises monotonically with scale. Comparing such failures against correct generations with matched semantic support, the distinguishing factor is not whether the correct concept is represented, but how its probability is distributed: correct generations concentrate mass on a single surface form, hallucinations disperse it across alternatives. The same sharpening asymmetry extends across multi-token generation and is detectable in pre-generation hidden states. Together, these results identify a single mechanism: instruction tuning sharpens answer commitment with scale, making helpfulness and confident hallucination two consequences of the same underlying disposition.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 46 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

35 of 46 references.

References · 35

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2024Qwen2.5 Technical Reportcited by 4,713
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  11. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  12. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
23 more
  1. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  2. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  3. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  4. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  5. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  6. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  7. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  8. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  9. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  10. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  11. 2024Calibrating Language Models with Adaptive Temperature Scalingcited by 57
  12. 2024MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMscited by 45
  13. 2025Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Modelscited by 39
  14. 2025Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuningcited by 33
  15. 2025Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistencycited by 30
  16. 2025Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answercited by 26
  17. 2024The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?cited by 24
  18. 2025Robust Hallucination Detection in LLMs via Adaptive Token Selectioncited by 19
  19. 2025Semantic Energy: Detecting LLM Hallucination Beyond Entropycited by 18
  20. 2025Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Mergingcited by 10
  21. 2026Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factualitycited by 10
  22. 2025Delusions of Large Language Modelscited by 4
  23. 2025First Hallucination Tokens Are Different from Conditional Onescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Reachability Is Not Realization: Tracing the Sources of LLM Benchmark Gainscited by 1