mlsanityData to 7 Oct 2026

LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations

arXiv 2410.02707 ↗First posted 3 Oct 2024cs.CL cs.AIcited by 230

Not on the map. Why?

Cited by 26 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) often produce errors, including factual inaccuracies, biases, and reasoning failures, collectively referred to as "hallucinations". Recent studies have demonstrated that LLMs' internal states encode information regarding the truthfulness of their outputs, and that this information can be utilized to detect errors. In this work, we show that the internal representations of LLMs encode much more information about truthfulness than previously recognized. We first discover that the truthfulness information is concentrated in specific tokens, and leveraging this property significantly enhances error detection performance. Yet, we show that such error detectors fail to generalize across datasets, implying that -- contrary to prior claims -- truthfulness encoding is not universal but rather multifaceted. Next, we show that internal representations can also be used for predicting the types of errors the model is likely to make, facilitating the development of tailored mitigation strategies. Lastly, we reveal a discrepancy between LLMs' internal encoding and external behavior: they may encode the correct answer, yet consistently generate an incorrect one. Taken together, these insights deepen our understanding of LLM errors from the model's internal perspective, which can guide future research on enhancing error analysis and mitigation.

Key facts. Cited by 230 papers in the map (data through 7 Oct 2026). Builds on 63 of its 95 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

63 of 95 references.

References · 63

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  10. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  11. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  12. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
38 more
  1. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  2. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  3. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  4. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  5. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  6. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  7. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  8. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  9. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  10. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  11. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  12. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  13. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  14. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  15. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  16. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  17. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  18. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  19. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  20. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 241
  21. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  22. 2023Fine-tuning Language Models for Factualitycited by 219
  23. 2023Gender bias and stereotypes in Large Language Modelscited by 185
  24. 2023Personality Traits in Large Language Modelscited by 166
  25. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 137
  26. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
  27. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
  28. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  29. 2019On Identifiability in Transformerscited by 115
  30. 2021Learning Compact Metrics for MTcited by 93
  31. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
  32. 2023TrueTeacher: Learning Factual Consistency Evaluation with Large Language Modelscited by 82
  33. 2024Truth is Universal: Robust Detection of Lies in LLMscited by 78
  34. 2023Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblockscited by 77
  35. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  36. 2023Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?cited by 62
  37. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 60
  38. 2023The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Modelscited by 54

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 89; 2026: 136.

230 papers.

Sort:

By area:

  1. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
  2. 2025Confidence Improves Self-Consistency in LLMscited by 111
  3. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 107
  4. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 87
  5. 2025Persona Features Control Emergent Misalignmentcited by 76
  6. 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languagescited by 57
  7. 2025Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verificationcited by 49
  8. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  9. 2025A Survey of Large Language Model Agents for Question Answeringcited by 46
  10. 2025Real-Time Detection of Hallucinated Entities in Long-Form Generationcited by 33
  11. 2025ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMscited by 31
  12. 2025Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinkingcited by 29
38 more
  1. 2025Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answercited by 26
  2. 2025The Illusion of Progress: Re-evaluating Hallucination Detection in LLMscited by 22
  3. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  4. 2025CoT-UQ: Improving Response-wise Uncertainty Quantification in LLMs with Chain-of-Thoughtcited by 20
  5. 2025Robust Hallucination Detection in LLMs via Adaptive Token Selectioncited by 19
  6. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
  7. 2025Hallucination Detection in LLMs with Topological Divergence on Attention Graphscited by 18
  8. 2025Too Consistent to Detect: A Study of Self-Consistent Errors in LLMscited by 18
  9. 2024VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understandingcited by 17
  10. 2025Probing for Arithmetic Errors in Language Modelscited by 17
  11. 2025Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Modelscited by 16
  12. 2025Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViTcited by 16
  13. 2025The Illusion of Certainty: Uncertainty Quantification for LLMs Fails under Ambiguitycited by 15
  14. 2025Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Deliverycited by 14
  15. 2025Large Language Models Encode Semantics and Alignment in Linearly Separable Representationscited by 14
  16. 2025Investigating task-specific prompts and sparse autoencoders for activation monitoringcited by 13
  17. 2026HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Tokencited by 13
  18. 2025The Geometry of Self-Verification in a Task-Specific Reasoning Modelcited by 12
  19. 2025Beyond Transcription: Mechanistic Interpretability in ASRcited by 12
  20. 2025Emergence of Linear Truth Encodings in Language Modelscited by 12
  21. 2025Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributionscited by 11
  22. 2025Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviorscited by 11
  23. 2025LLM Microscope: What Model Internals Reveal About Answer Correctness and Context Utilizationcited by 11
  24. 2025Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generationcited by 10
  25. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  26. 2025Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigationcited by 9
  27. 2025The Geometries of Truth Are Orthogonal Across Taskscited by 9
  28. 2025LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblancecited by 9
  29. 2025Learning to Instruct for Visual Instruction Tuningcited by 8
  30. 2025BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMscited by 8
  31. 2025HiProbe-VAD: Video Anomaly Detection via Hidden States Probing in Tuning-Free Multimodal LLMscited by 8
  32. 2026Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Controlcited by 8
  33. 2024Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialoguescited by 7
  34. 2025TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Modelscited by 7
  35. 2025Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulnesscited by 7
  36. 2026BaseCal: Unsupervised Confidence Calibration via Base Model Signalscited by 7
  37. 2026TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphscited by 7
  38. 2026The Truthfulness Spectrum Hypothesiscited by 7

50 most cited of 230 All 230 (JSON)