mlsanityData to 7 Oct 2026

Understanding Why Language Models Hallucinate: Testing Reasoning Against Priors

arXiv 2607.00447 ↗First posted 1 Jul 2026cs.CLcited by 1

On the map: LLM reasoning → Hallucination detection and mitigation

Abstract (arXiv)

Large language models often produce hallucinated answers that violate prompt-level constraints. A key diagnostic question is whether these failures reflect missing knowledge, or whether the model has the relevant information but follows the wrong inference path. We study this phenomenon as inference misalignment: a mismatch between the answer supported by the prompt and the answer favored by statistically salient latent associations. We formalize this view with a latent key-task model, in which pretraining-frequency imbalance can cause a shortcut path to dominate the constraint-sensitive path and induce positive inference loss. The framework predicts two failure modes: task-retrieval bias in entity disambiguation and key-selection bias in action choice. We introduce TrapQA, a controlled diagnostic testbed with two components. ScientistQA tests disambiguation among similar scientists with supplementary factual probes, while Real-Life Constrained QA tests everyday constraint following under salient shortcuts. Our results show that hallucination can arise from biased latent inference rather than absent knowledge alone.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 46 of its 79 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

46 of 79 references.

References · 46

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  6. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  7. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  8. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  9. 2024DeepSeek-V3 Technical Reportcited by 724
  10. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  11. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
  12. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
34 more
  1. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
  2. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  3. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  4. 2024Measuring short-form factuality in large language modelscited by 345
  5. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  6. 2023How Language Model Hallucinations Can Snowballcited by 310
  7. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 285
  8. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  9. 2025Why Language Models Hallucinatecited by 205
  10. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168
  11. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  12. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 158
  13. 2024Long-form factuality in large language modelscited by 146
  14. 2023Calibrated Language Models Must Hallucinatecited by 114
  15. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 106
  16. 2023WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferencescited by 100
  17. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 85
  18. 2023Why Does ChatGPT Fall Short in Providing Truthful Answers?cited by 54
  19. 2024FLAME: Factuality-Aware Alignment for Large Language Modelscited by 46
  20. 2024Learning or Self-aligning? Rethinking Instruction Fine-tuningcited by 43
  21. 2023HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translationcited by 35
  22. 2023Evaluating Hallucinations in Chinese Large Language Modelscited by 33
  23. 2024WildHallucinations: Evaluating Long-form Factuality in LLMs with Real-World Entity Queriescited by 26
  24. 2025TruthRL: Incentivizing Truthful LLMs via Reinforcement Learningcited by 20
  25. 2025H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMscited by 19
  26. 2025Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associationscited by 14
  27. 2022TRUE: Re-evaluating Factual Consistency Evaluationcited by 7
  28. 2025SeSE: Black-Box Uncertainty Quantification for Large Language Models Based on Structural Information Theorycited by 7
  29. 2025Harnessing RLHF for Robust Unanswerability Recognition and Trustworthy Response Generation in LLMscited by 3
  30. 2026Hallucination Is Linearly Decodable from Mid-Layer Hidden States in Quantized LLMscited by 3
  31. 2025Measuring the Impact of Lexical Training Data Coverage on Hallucination Detection in Large Language Modelscited by 2
  32. 2026Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinationscited by 2
  33. 2026PretrainRL: Alleviating Factuality Hallucination of Large Language Models at the Beginningcited by 1
  34. 2026OpenHalDet: A Unified Benchmark for Hallucination Detection across Diverse Generation Scenarioscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?
  2. 2026The Alignment Paradox: How Post-Training Amplifies Confident Hallucinations in Language Models
  3. 2025Hallucinate Less by Thinking More: Aspect-Based Causal Abstention for Large Language Models
  4. 2026Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs
  5. 2025Reasoning Large Language Model Errors Arise from Hallucinating Critical Problem Features
  6. 2026From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data
  7. 2026Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations
  8. 2025Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspective