mlsanityData to 14 Oct 2026

Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models

arXiv 2411.14257 ↗First posted 21 Nov 2024cs.CL cs.AI cs.LGcited by 115

On the map: Mechanistic interpretability → Hallucination detection from model internals

Abstract (arXiv)

Hallucinations in large language models are a widespread problem, yet the mechanisms behind whether models will hallucinate are poorly understood, limiting our ability to solve this problem. Using sparse autoencoders as an interpretability tool, we discover that a key part of these mechanisms is entity recognition, where the model detects if an entity is one it can recall facts about. Sparse autoencoders uncover meaningful directions in the representation space, these detect whether the model recognizes an entity, e.g. detecting it doesn't know about an athlete or a movie. This suggests that models can have self-knowledge: internal representations about their own capabilities. These directions are causally relevant: capable of steering the model to refuse to answer questions about known entities, or to hallucinate attributes of unknown entities when it would otherwise refuse. We demonstrate that despite the sparse autoencoders being trained on the base model, these directions have a causal effect on the chat model's refusal behavior, suggesting that chat finetuning has repurposed this existing mechanism. Furthermore, we provide an initial exploration into the mechanistic role of these directions in the model, finding that they disrupt the attention of downstream heads that typically move entity attributes to the final token.

Key facts. Cited by 115 papers in the map (data through 14 Oct 2026). Placed in Hallucination detection from model internals, one of 13 directions in Mechanistic interpretability. Builds on 33 of its 56 references that are in the map. First posted on arXiv 21 Nov 2024.

Builds on

33 of 56 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Do Androids Know They're Only Dreaming of Electric Sheep?28 Dec

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  5. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  6. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  7. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  9. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  10. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  11. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  12. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
20 more
  1. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  2. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  3. 2024Scaling and evaluating sparse autoencoderscited by 553
  4. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  5. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  6. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  7. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
  8. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  9. 2024How to use and interpret activation patchingcited by 214
  10. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
  11. 2023Linear Representations of Sentiment in Large Language Modelscited by 163
  12. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 149
  13. 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 115
  14. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 87
  15. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  16. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  17. 2024Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMscited by 42
  18. 2024Estimating Knowledge in Large Language Models Without Generating a Single Tokencited by 40
  19. 2024Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinationscited by 38
  20. 2019JumpReLU: A Retrofit Defense Strategy for Adversarial Attackscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 62; 2026: 53.

115 papers.

Sort:

By area:

  1. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 76
  2. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 50
  3. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  4. 2025Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?cited by 38
  5. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 38
  6. 2025SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Modelscited by 34
  7. 2025Real-Time Detection of Hallucinated Entities in Long-Form Generationcited by 34
  8. 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Modelscited by 29
  9. 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 26
  10. 2025The Dual-Route Model of Inductioncited by 24
  11. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 24
  12. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 21
38 more
  1. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 21
  2. 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 21
  3. 2025Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Modelscited by 18
  4. 2025Your AI, Not Your View: The Bias of LLMs in Investment Analysiscited by 17
  5. 2025On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedycited by 14
  6. 2025The Geometry of Self-Verification in a Task-Specific Reasoning Modelcited by 13
  7. 2025Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoderscited by 13
  8. 2025EasySteer: A Unified Framework for High-Performance and Extensible LLM Steeringcited by 13
  9. 2025Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generationcited by 12
  10. 2025Emergence of Linear Truth Encodings in Language Modelscited by 12
  11. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?cited by 12
  12. 2026PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebracited by 12
  13. 2025Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviorscited by 11
  14. 2025Unsupervised Elicitation of Language Modelscited by 11
  15. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  16. 2025When Truthful Representations Flip Under Deceptive Instructions?cited by 10
  17. 2025Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinementcited by 10
  18. 2025Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoderscited by 10
  19. 2025SAGE: An Agentic Explainer Framework for Interpreting SAE Features in Language Modelscited by 10
  20. 2025Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMscited by 9
  21. 2025A Single Direction of Truth: An Observer Model's Linear Residual Probe Exposes and Steers Contextual Hallucinationscited by 9
  22. 2025CyberV: Cybernetics for Test-time Scaling in Video Understandingcited by 8
  23. 2025Mechanistic Indicators of Understanding in Large Language Modelscited by 8
  24. 2025PredictaBoard: Benchmarking LLM Score Predictabilitycited by 7
  25. 2025EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Modelscited by 7
  26. 2025Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoderscited by 7
  27. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoderscited by 7
  28. 2025CDF-RAG: Causal Dynamic Feedback for Adaptive Retrieval-Augmented Generationcited by 6
  29. 2025Fresh in memory: Training-order recency is linearly encoded in language model activationscited by 6
  30. 2025Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learningcited by 6
  31. 2026Sparse Autoencoders are Capable LLM Jailbreak Mitigatorscited by 6
  32. 2026Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoderscited by 6
  33. 2025Position: Stop Acting Like Language Model Agents Are Normal Agentscited by 5
  34. 2025Agent Identity Evals: Measuring Agentic Identitycited by 5
  35. 2025Mechanistic Interpretability of Code Correctness in LLMs via Sparse Autoencoderscited by 5
  36. 2025SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucinationcited by 5
  37. 2025RePPL: Recalibrating Perplexity by Uncertainty in Semantic Propagation and Language Generation for Explainable QA Hallucination Detectioncited by 4
  38. 2025Cross-Layer Attention Probing for Fine-Grained Hallucination Detectioncited by 4

50 most cited of 115 All 115 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations
  2. 2026A Geometric Analysis of Small-sized Language Model Hallucinations
  3. 2026HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token
  4. 2026External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing
  5. 2026Unmasking Hallucinations: A Causal Graph-Attention Perspective on Factual Reliability in Large Language Models
  6. 2023Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?
  7. 2025Steering LVLMs via Sparse Autoencoder for Hallucination Mitigation
  8. 2026Predictive Coding and Information Bottleneck for Hallucination Detection in Large Language Models