mlsanityData to 14 Oct 2026

Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models

arXiv 2505.13143 ↗First posted 19 May 2025cs.CYcited by 20

On the map: LLM reasoning → Hallucination detection and mitigation

Abstract (arXiv)

The development of Reasoning Large Language Models (RLLMs) has significantly improved multi-step reasoning capabilities, but it has also made hallucination problems more frequent and harder to eliminate. While existing approaches mitigate hallucinations through external knowledge integration, model parameter analysis, or self-verification, they often fail to capture how hallucinations emerge and evolve across the reasoning chain. In this work, we study the causality of hallucinations under constrained knowledge domains by auditing the Chain-of-Thought (CoT) trajectory and assessing the model's cognitive confidence in potentially erroneous or biased claims. Our analysis reveals that in long-CoT settings, RLLMs can iteratively reinforce biases and errors through flawed reflective reasoning, eventually leading to hallucinated reasoning paths. Surprisingly, even direct interventions at the origin of hallucinations often fail to reverse their effects, as reasoning chains exhibit 'chain disloyalty' -- a resistance to correction and a tendency to preserve flawed logic. Furthermore, we show that existing hallucination detection methods are less reliable and interpretable than previously assumed in complex reasoning scenarios. Unlike methods such as circuit tracing that require access to model internals, our black-box auditing approach supports interpretable long-chain hallucination attribution, offering better generalizability and practical utility. Our code is available at: https://github.com/Winnie-Lian/AHa_Meta_Cognitive

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 23 of its 76 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

23 of 76 references.

References · 23

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  3. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  4. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  5. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  6. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  7. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  8. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  9. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 269
  10. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 262
  11. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 170
  12. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 169
11 more
  1. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 153
  2. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 83
  3. 2023A Survey of Reasoning with Foundation Modelscited by 70
  4. 2023Natural Language Reasoning, A Surveycited by 61
  5. 2025Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Modelscited by 27
  6. 2023Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuningcited by 26
  7. 2025On the Emergence of Thinking in LLMs I: Searching for the Right Intuitioncited by 22
  8. 2025A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworkscited by 21
  9. 2024Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Informationcited by 10
  10. 2025Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instructioncited by 10
  11. 2024Lachesis: Predicting LLM Inference Accuracy using Structural Properties of Reasoning Pathscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 16.

20 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  2. 2026FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insightscited by 16
  3. 2025Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Modelscited by 14
  4. 2026Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware Decodingcited by 12
  5. 2026ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spacescited by 8
  6. 2025Agentic Services Computingcited by 6
  7. 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forwardcited by 6
  8. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 6
  9. 2026Streaming Hallucination Detection in Long Chain-of-Thought Reasoningcited by 5
  10. 2026Fine-Tuning Language Models to Know What They Knowcited by 2
  11. 2026Self-Reference in Large Language Models: The Introspection Threshold for Recursive Self-Improvementcited by 2
  12. 2026MIDAS: Multi-Image Dispersion and Semantic Reconstruction for Jailbreaking MLLMscited by 1
8 more
  1. 2026Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoningcited by 1
  2. 2026Enhancing LLM Metacognition via Cognitive Pairwise Trainingcited by 1
  3. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
  4. 2026ClueTracer: Question-to-Vision Clue Tracing for Training-Free Hallucination Suppression in Multimodal Reasoningcited by 0
  5. 2026ReBeCA: Unveiling Interpretable Behavior Hierarchy behind the Iterative Self-Reflection of Language Models with Causal Analysiscited by 0
  6. 2026Disentangling Deception and Hallucination Failures in LLMscited by 0
  7. 2026AI Knows What's Wrong But Cannot Fix It: Helicoid Dynamics in Frontier LLMs Under High-Stakes Decisionscited by 0
  8. 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Studycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspective
  2. 2025Hallucination Detection via Internal States and Structured Reasoning Consistency in Large Language Models
  3. 2026PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinations
  4. 2025Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluation
  5. 2026Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models
  6. 2026HalluGuard: Demystifying Data-Driven and Reasoning-Driven Hallucinations in LLMs
  7. 2026HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models
  8. 2025Triggering Hallucinations in LLMs: A Quantitative Study of Prompt-Induced Hallucination in Large Language Models