mlsanityData to 7 Oct 2026

Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States

arXiv 2402.09733 ↗First posted 15 Feb 2024cs.CLcited by 50

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) can make up answers that are not real, and this is known as hallucination. This research aims to see if, how, and to what extent LLMs are aware of hallucination. More specifically, we check whether and how an LLM reacts differently in its hidden states when it answers a question right versus when it hallucinates. To do this, we introduce an experimental framework which allows examining LLM's hidden states in different hallucination situations. Building upon this framework, we conduct a series of experiments with language models in the LLaMA family (Touvron et al., 2023). Our empirical findings suggest that LLMs react differently when processing a genuine response versus a fabricated one. We then apply various model interpretation techniques to help understand and explain the findings better. Moreover, informed by the empirical observations, we show great potential of using the guidance derived from LLM's hidden representation space to mitigate hallucination. We believe this work provides insights into how LLMs produce hallucinated answers and how to make them occur less often.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 15 Feb 2024.

Builds on

18 of 35 references.

References · 18

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  7. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  8. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  9. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  10. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  11. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  12. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
6 more
  1. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  2. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 235
  3. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
  4. 2023Complex Claim Verification with Evidence Retrieved in the Wildcited by 110
  5. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
  6. 2020Controlled Hallucinations: Learning to Generate Faithfully from Noisy Datacited by 83

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 24; 2026: 10.

50 papers.

Sort:

By area:

  1. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
  2. 2024Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approachcited by 73
  3. 2024Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluationcited by 56
  4. 2024Efficiently Scaling LLM Reasoning with Certaindexcited by 56
  5. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  6. 2024Safety-Aware Fine-Tuning of Large Language Modelscited by 33
  7. 2024Do LLMs "know" internally when they follow instructions?cited by 25
  8. 2024Reference-free Hallucination Detection for Large Vision-Language Modelscited by 23
  9. 2025Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputscited by 15
  10. 2024DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metricscited by 13
  11. 2025Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuitscited by 12
  12. 2024FactCheckmate: Preemptively Detecting and Mitigating Hallucinations in LMscited by 11
38 more
  1. 2024When Context Leads but Parametric Memory Follows in Large Language Modelscited by 10
  2. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  3. 2025Shadows in the Attention: Contextual Perturbation and Representation Drift in the Dynamics of Hallucination in LLMscited by 9
  4. 2026Quantifying and Mitigating Self-Preference Bias of LLM Judgescited by 9
  5. 2024LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM statescited by 8
  6. 2025Bridging Visualization and Optimization: Multimodal Large Language Models on Graph-Structured Combinatorial Optimizationcited by 6
  7. 2025Explainable XR: Understanding User Behaviors of XR Environments using LLM-assisted Analytics Frameworkcited by 6
  8. 2025JAM: Controllable and Responsible Text Generation via Causal Reasoning and Latent Vector Manipulationcited by 5
  9. 2025Do Large Language Models (Really) Need Statistical Foundations?cited by 5
  10. 2025CoRE: Enhancing Metacognition with Label-free Self-evaluation in LRMscited by 5
  11. 2026ReasoningBomb: A Stealthy Denial-of-Service Attack by Inducing Pathologically Long Reasoning in Large Reasoning Modelscited by 5
  12. 2024Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers?cited by 4
  13. 2025Monitoring Decoding: Mitigating Hallucination via Evaluating the Factuality of Partial Response during Generationcited by 4
  14. 2025LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMscited by 4
  15. 2025Language Surgery in Multilingual Large Language Modelscited by 4
  16. 2024Medico: Towards Hallucination Detection and Correction with Multi-source Evidence Fusioncited by 3
  17. 2025Can Large Language Models Be Trusted as Evolutionary Optimizers for Network-Structured Combinatorial Problems?cited by 3
  18. 2025Latte: Transfering LLMs` Latent-level Knowledge for Few-shot Tabular Learningcited by 3
  19. 2025Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddingscited by 3
  20. 2025Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitionscited by 3
  21. 2025What are Models Thinking about? Understanding Large Language Model Hallucinations "Psychology" through Model Inner State Analysiscited by 2
  22. 2025Finetune-RAG: Fine-Tuning Language Models to Resist Hallucination in Retrieval-Augmented Generationcited by 2
  23. 2025From Hallucinations to Jailbreaks: Rethinking the Vulnerability of Large Foundation Modelscited by 2
  24. 2025Shaking to Reveal: Perturbation-Based Detection of LLM Hallucinationscited by 2
  25. 2025HIDE and Seek: Detecting Hallucinations in Language Models via Decoupled Representationscited by 2
  26. 2025Shapley Uncertainty in Natural Language Generationcited by 2
  27. 2026Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicitycited by 1
  28. 2026RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integrationcited by 1
  29. 2026Shared Doubt: Zero-Shot Cross-Lingual Confidence Estimation for Language Modelscited by 1
  30. 2024AILS-NTUA at SemEval-2024 Task 6: Efficient model tuning for hallucination detection and analysiscited by 0
  31. 2024Interweaving Memories of a Siamese Large Language Modelcited by 0
  32. 2025MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinationscited by 0
  33. 2025Toward Better Generalisation in Uncertainty Estimators: Leveraging Data-Agnostic Featurescited by 0
  34. 2026On the Tip of the Tongue: Why LLMs Hallucinate Answers They Can Decodecited by 0
  35. 2026Entropy Distribution as a Fingerprint for Hallucinations in Generative Modelscited by 0
  36. 2026TAROT: Task-Adaptive Refinement of LLM-prior Graphs for Few-shot Tabular Learningcited by 0
  37. 2026Geometry of Divergence: Tracking Hidden-State Trajectories for Adaptive Multi-Turn Reasoningcited by 0
  38. 2026Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generationcited by 0