mlsanityData to 7 Oct 2026

HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders

arXiv 2604.16430 ↗First posted 6 Apr 2026cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are powerful and widely adopted, but their practical impact is limited by the well-known hallucination phenomenon. While recent hallucination detection methods have made notable progress, we find most of them overlook the dynamic nature and underlying mechanisms of it. To address this gap, we propose HalluSAE, a phase transition-inspired framework that models hallucination as a critical shift in the model's latent dynamics. By modeling the generation process as a trajectory through a potential energy landscape, HalluSAE identifies critical transition zones and attributes factual errors to specific high-energy sparse features. Our approach consists of three stages: (1) Potential Energy Empowered Phase Zone Localization via sparse autoencoders and a geometric potential energy metric; (2) Hallucination-related Sparse Feature Attribution using contrastive logit attribution; and (3) Probing-based Causal Hallucination Detection through linear probes on disentangled features. Extensive experiments on Gemma-2-9B demonstrate that HalluSAE achieves state-of-the-art hallucination detection performance.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 46 references that are in the map. First posted on arXiv 6 Apr 2026.

Builds on

39 of 46 references.

References · 39

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  5. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  6. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  7. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  8. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  9. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  10. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  11. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  12. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
27 more
  1. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  2. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  3. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  4. 2024Scaling and evaluating sparse autoencoderscited by 530
  5. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  6. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  7. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  8. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  9. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  10. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  11. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  12. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
  13. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  14. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  15. 2023Chainpoll: A high efficacy method for LLM hallucination detectioncited by 39
  16. 2025The Law of Knowledge Overshadowing: Towards Understanding, Predicting, and Preventing LLM Hallucinationcited by 22
  17. 2025Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Modelscited by 16
  18. 2024Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Modelscited by 13
  19. 2025Language Model Uncertainty Quantification with Attention Chaincited by 13
  20. 2025SAFE: A Sparse Autoencoder-Based Framework for Robust Query Enrichment and Hallucination Mitigation in LLMscited by 7
  21. 2025PrivacyScalpel: Enhancing LLM Privacy via Interpretable Feature Intervention with Sparse Autoencoderscited by 6
  22. 2025Steering LVLMs via Sparse Autoencoder for Hallucination Mitigationcited by 6
  23. 2025Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoderscited by 6
  24. 2025HARP: Hallucination Detection via Reasoning Subspace Projectioncited by 4
  25. 2025RePPL: Recalibrating Perplexity by Uncertainty in Semantic Propagation and Language Generation for Explainable QA Hallucination Detectioncited by 3
  26. 2025SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucinationcited by 3
  27. 2025A Graph Signal Processing Framework for Hallucination Detection in Large Language Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Temporal Multi-Signal Fusion for Token-Level Hallucination Detectioncited by 1
  2. 2026DeMTS: Denoising Trajectories as Multivariate Time Series for Hallucination Detection in Diffusion Language Modelscited by 0
  3. 2026Active Budget Can Kill Sensitivity: Diagnosing and Repairing TopK Sparse Autoencoder Reliabilitycited by 0