mlsanityData to 14 Oct 2026

The Internal State of an LLM Knows When It's Lying

arXiv 2304.13734 ↗First posted 26 Apr 2023cs.CL cs.AI cs.LGcited by 659

Not on the map. Why?

Cited by 71 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While Large Language Models (LLMs) have shown exceptional performance in various tasks, one of their most prominent drawbacks is generating inaccurate or false information with a confident tone. In this paper, we provide evidence that the LLM's internal state can be used to reveal the truthfulness of statements. This includes both statements provided to the LLM, and statements that the LLM itself generates. Our approach is to train a classifier that outputs the probability that a statement is truthful, based on the hidden layer activations of the LLM as it reads or generates the statement. Experiments demonstrate that given a set of test sentences, of which half are true and half false, our trained classifier achieves an average of 71% to 83% accuracy labeling which sentences are true versus false, depending on the LLM base model. Furthermore, we explore the relationship between our classifier's performance and approaches based on the probability assigned to the sentence by the LLM. We show that while LLM-assigned sentence probability is related to sentence truthfulness, this probability is also dependent on sentence length and the frequencies of words in the sentence, resulting in our trained classifier providing a more reliable approach to detecting truthfulness, highlighting its potential to enhance the reliability of LLM-generated content and its practical applicability in real-world scenarios.

Key facts. Cited by 659 papers in the map (data through 14 Oct 2026). Builds on 15 of its 20 references that are in the map. First posted on arXiv 26 Apr 2023.

Builds on

15 of 20 references.

References · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  8. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  9. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  10. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  11. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
  12. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 287
3 more
  1. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  2. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  3. 2022GPT Takes the Bar Examcited by 65

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 57; 2024: 153; 2025: 174; 2026: 275.

659 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  2. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  3. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  4. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 418
  5. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  6. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  7. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
  8. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
  9. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
  10. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
  11. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
  12. 2023Fine-tuning Language Models for Factualitycited by 221
38 more
  1. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
  2. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  3. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 202
  4. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 188
  5. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 180
  6. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 179
  7. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 160
  8. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
  9. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
  10. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 127
  11. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 124
  12. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
  13. 2024To Believe or Not to Believe Your LLMcited by 120
  14. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 117
  15. 2023Do Language Models Know When They're Hallucinating References?cited by 116
  16. 2023Calibrated Language Models Must Hallucinatecited by 115
  17. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 115
  18. 2025Confidence Improves Self-Consistency in LLMscited by 113
  19. 2024What Large Language Models Know and What People Think They Knowcited by 110
  20. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Costcited by 109
  21. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 104
  22. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 104
  23. 2023Deception Abilities Emerged in Large Language Modelscited by 101
  24. 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretabilitycited by 101
  25. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 100
  26. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 100
  27. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 99
  28. 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 97
  29. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 95
  30. 2024Large language model validity via enhanced conformal prediction methodscited by 92
  31. 2024TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Spacecited by 91
  32. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 88
  33. 2025LLMs Encode Harmfulness and Refusal Separatelycited by 88
  34. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 87
  35. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 87
  36. 2024Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Modelscited by 86
  37. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 83
  38. 2024Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluationcited by 80

50 most cited of 659 All 659 (JSON)