mlsanityData to 7 Oct 2026

What are Models Thinking about? Understanding Large Language Model Hallucinations "Psychology" through Model Inner State Analysis

arXiv 2502.13490 ↗First posted 19 Feb 2025cs.CL cs.AIcited by 2

On the map: LLM reasoning → Hallucination detection and mitigation

Abstract (arXiv)

Large language model (LLM) systems suffer from the models' unstable ability to generate valid and factual content, resulting in hallucination generation. Current hallucination detection methods heavily rely on out-of-model information sources, such as RAG to assist the detection, thus bringing heavy additional latency. Recently, internal states of LLMs' inference have been widely used in numerous research works, such as prompt injection detection, etc. Considering the interpretability of LLM internal states and the fact that they do not require external information sources, we introduce such states into LLM hallucination detection. In this paper, we systematically analyze different internal states' revealing features during inference forward and comprehensively evaluate their ability in hallucination detection. Specifically, we cut the forward process of a large language model into three stages: understanding, query, generation, and extracting the internal state from these stages. By analyzing these states, we provide a deep understanding of why the hallucinated content is generated and what happened in the internal state of the models. Then, we introduce these internal states into hallucination detection and conduct comprehensive experiments to discuss the advantages and limitations.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 26 of its 60 references that are in the map. First posted on arXiv 19 Feb 2025.

Builds on

26 of 60 references.

References · 26

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2023BloombergGPT: A Large Language Model for Financecited by 923
  4. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  5. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  6. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  7. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  8. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  9. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  10. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
  11. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  12. 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 241
14 more
  1. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168
  2. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
  3. 2022Analyzing Transformers in Embedding Spacecited by 131
  4. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  5. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 87
  6. 2024Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Modelscited by 84
  7. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 80
  8. 2024Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Basescited by 76
  9. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  10. 2024Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden Statescited by 50
  11. 2024In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigationcited by 43
  12. 2024Detecting Hallucinations in Large Language Model Generation: A Token Probability Approachcited by 40
  13. 2024InternalInspector I2: Robust Confidence Estimation in LLMs through Internal Statescited by 29
  14. 2024Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Costcited by 5

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Shadows in the Attention: Contextual Perturbation and Representation Drift in the Dynamics of Hallucination in LLMscited by 9
  2. 2025Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Modelscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2026External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing
  2. 2025Theoretical Foundations and Mitigation of Hallucination in Large Language Models
  3. 2025(Im)possibility of Automated Hallucination Detection in Large Language Models
  4. 2025Hallucination Detection via Internal States and Structured Reasoning Consistency in Large Language Models
  5. 2024Investigating the Role of Prompting and External Tools in Hallucination Rates of Large Language Models
  6. 2026HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token
  7. 2025TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Intervention
  8. 2025Revisiting Hallucination Detection with Effective Rank-based Uncertainty