mlsanityData to 14 Oct 2026

Unveiling Decision-Making in LLMs for Text Classification : Extraction of influential and interpretable concepts with Sparse Autoencoders

arXiv 2506.23951 ↗First posted 30 Jun 2025cs.CLcited by 3

On the map: Mechanistic interpretability → SAE features and their applications

Abstract (arXiv)

Sparse Autoencoders (SAEs) have been successfully used to probe Large Language Models (LLMs) and extract interpretable concepts from their internal representations. These concepts are linear combinations of neuron activations that correspond to human-interpretable features. In this paper, we investigate the effectiveness of SAE-based explainability approaches for sentence classification, a domain where such methods have not been extensively explored. We present a novel SAE-based model ClassifSAE tailored for text classification, leveraging a specialized classifier head and incorporating an activation rate sparsity loss. We benchmark this architecture against established methods such as ConceptShap, Independent Component Analysis, HI-Concept and a standard TopK-SAE baseline. Our evaluation covers several classification benchmarks and backbone LLMs. We further enrich our analysis with two novel metrics for measuring the precision of concept-based explanations, using an external sentence encoder. Our empirical results show that ClassifSAE improves both the causality and interpretability of the extracted features.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability. Builds on 32 of its 53 references that are in the map. First posted on arXiv 30 Jun 2025.

Builds on

32 of 53 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Sparse Autoencoder Features for Classifications and Transferability17 Feb
  2. 2024Improving Dictionary Learning with Gated Sparse Autoencoders24 Apr
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Models15 Sep

Other references · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  5. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  7. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  8. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  9. 2018Sanity Checks for Saliency Mapscited by 1,156
  10. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  11. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  12. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
17 more
  1. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  2. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  3. 2019SemEval-2017 Task 4: Sentiment Analysis in Twittercited by 210
  4. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 207
  5. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
  6. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 178
  7. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  8. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
  9. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 111
  10. 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 98
  11. 2020Guided Variational Autoencoder for Disentanglement Learningcited by 57
  12. 2021A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Conceptscited by 33
  13. 2023Sparse Linear Concept Discovery Modelscited by 22
  14. 2023COCKATIEL: COntinuous Concept ranKed ATtribution with Interpretable ELements for explaining neural net classifiers on NLP taskscited by 16
  15. 2024MCPNet: An Interpretable Classifier via Multi-Level Concept Prototypescited by 10
  16. 2024Evaluating Readability and Faithfulness of Concept-based Explanationscited by 10
  17. 2024Adaptive Sparse Allocation with Mutual Choice & Feature Choice Sparse Autoencoderscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026When Benchmarks Lie: Evaluating Malicious Prompt Classifiers Under True Distribution Shiftcited by 11
  2. 2026Decoder-Preserving Sparse Autoencoders: Which Readouts Survive Sparse Compression?cited by 0
  3. 2026J-Miner: Recovering the Decision Logic of Fine-Tuned LLM Classifiers as Compact Rulescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models
  2. 2024Automatically Interpreting Millions of Features in Large Language Models
  3. 2025Evaluating SAE interpretability without explanations
  4. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probing
  5. 2026Concept Component Analysis: A Principled Approach for Concept Extraction in LLMs
  6. 2025Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations
  7. 2026Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility
  8. 2025Interpreting and Steering LLMs with Mutual Information-based Explanations on Sparse Autoencoders