mlsanityData to 7 Oct 2026

LLM Internal States Reveal Hallucination Risk Faced With a Query

arXiv 2407.03282 ↗First posted 3 Jul 2024cs.CLcited by 80

Not on the map. Why?

Abstract (arXiv)

The hallucination problem of Large Language Models (LLMs) significantly limits their reliability and trustworthiness. Humans have a self-awareness process that allows us to recognize what we don't know when faced with queries. Inspired by this, our paper investigates whether LLMs can estimate their own hallucination risk before response generation. We analyze the internal mechanisms of LLMs broadly both in terms of training data sources and across 15 diverse Natural Language Generation (NLG) tasks, spanning over 700 datasets. Our empirical analysis reveals two key insights: (1) LLM internal states indicate whether they have seen the query in training data or not; and (2) LLM internal states show they are likely to hallucinate or not regarding the query. Our study explores particular neurons, activation layers, and tokens that play a crucial role in the LLM perception of uncertainty and hallucination risk. By a probing estimator, we leverage LLM self-assessment, achieving an average hallucination estimation accuracy of 84.32% at run time.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). Builds on 31 of its 50 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

31 of 50 references.

References · 31

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  6. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  7. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  8. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  9. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  10. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  11. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  12. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
19 more
  1. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  2. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 241
  3. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  4. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
  5. 2023Towards Mitigating Hallucination in Large Language Models via Self-Reflectioncited by 81
  6. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  7. 2024Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approachcited by 73
  8. 2021Towards Few-Shot Fact-Checking via Perplexitycited by 63
  9. 2019Out-of-domain Detection for Natural Language Understanding in Dialog Systemscited by 62
  10. 2023Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?cited by 62
  11. 2023LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Constructioncited by 57
  12. 2022Entity Cloze By Date: What LMs Know About Unseen Entitiescited by 55
  13. 2023On Early Detection of Hallucinations in Factual Question Answeringcited by 54
  14. 2022RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Groundingcited by 47
  15. 2024Distinguishing the Knowable from the Unknowable with Language Modelscited by 45
  16. 2024Estimating Knowledge in Large Language Models Without Generating a Single Tokencited by 39
  17. 2019Out-of-Domain Detection for Low-Resource Text Classification Taskscited by 36
  18. 2023ALCUNA: Large Language Models Meet New Knowledgecited by 35
  19. 2022Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learningcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 42; 2026: 30.

80 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2024Attention Heads of Large Language Models: A Surveycited by 65
  3. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  4. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  5. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  6. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 35
  7. 2025Real-Time Detection of Hallucinated Entities in Long-Form Generationcited by 33
  8. 2025SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signalscited by 29
  9. 2024A Survey on the Honesty of Large Language Modelscited by 21
  10. 2025Semantic Volume: Quantifying and Detecting both External and Internal Uncertainty in LLMscited by 19
  11. 2025Robust Hallucination Detection in LLMs via Adaptive Token Selectioncited by 19
  12. 2025The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hackingcited by 18
38 more
  1. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17
  2. 2025Emergent Response Planning in LLMscited by 17
  3. 2024Distinguishing Ignorance from Error in LLM Hallucinationscited by 14
  4. 2025Depth Gives a False Sense of Privacy: LLM Internal States Inversioncited by 14
  5. 2024UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Modelscited by 13
  6. 2025Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviorscited by 11
  7. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  8. 2025BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMscited by 8
  9. 2025Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulnesscited by 7
  10. 2025Hallucinated Span Detection with Multi-View Attention Featurescited by 6
  11. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 6
  12. 2025HACK: Hallucinations Along Certainty and Knowledge Axescited by 6
  13. 2026FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAGcited by 6
  14. 2026LLMs Should Express Uncertainty Explicitlycited by 6
  15. 2025Representation Consistency for Accurate and Coherent LLM Answer Aggregationcited by 5
  16. 2025Learning to Reason for Hallucination Span Detectioncited by 5
  17. 2026Gradient-Stable Attention Heads Signal LLM Correctnesscited by 5
  18. 2025Knowing When to Stop: Efficient Context Processing via Latent Sufficiency Signalscited by 4
  19. 2025Language Surgery in Multilingual Large Language Modelscited by 4
  20. 2026Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascadecited by 4
  21. 2024Tell Me What You Don't Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editingcited by 3
  22. 2025RePPL: Recalibrating Perplexity by Uncertainty in Semantic Propagation and Language Generation for Explainable QA Hallucination Detectioncited by 3
  23. 2025Towards Harmonized Uncertainty Estimation for Large Language Modelscited by 3
  24. 2025Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledgecited by 3
  25. 2025Cleanse: Uncertainty Estimation Approach Using Clustering-based Semantic Consistency in LLMscited by 3
  26. 2025Test of Time: Rethinking Temporal Signal of Benchmark Contaminationcited by 3
  27. 2025Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Modelscited by 3
  28. 2026Automatic Layer Selection for Hallucination Detectioncited by 3
  29. 2026How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMscited by 3
  30. 2026Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden Statescited by 3
  31. 2024Probing LLM Hallucination from Within: Perturbation-Driven Approach via Internal Knowledgecited by 2
  32. 2025What are Models Thinking about? Understanding Large Language Model Hallucinations "Psychology" through Model Inner State Analysiscited by 2
  33. 2025High-Dimensional Interlingual Representations of Large Language Modelscited by 2
  34. 2025Factual Self-Awareness in Language Models: Representation, Robustness, and Scalingcited by 2
  35. 2025HIDE and Seek: Detecting Hallucinations in Language Models via Decoupled Representationscited by 2
  36. 2025ISACL: Internal State Analyzer for Copyrighted Training Data Leakagecited by 2
  37. 2026When is Your LLM Steerable?cited by 2
  38. 2025HalluCounter: Reference-free LLM Hallucination Detection in the Wild!cited by 1

50 most cited of 80 All 80 (JSON)