mlsanityData to 7 Oct 2026

Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning

arXiv 2405.12241 ↗First posted 17 May 2024cs.LG cs.AIcited by 64

Not on the map. Why?

Abstract (arXiv)

Identifying the features learned by neural networks is a core challenge in mechanistic interpretability. Sparse autoencoders (SAEs), which learn a sparse, overcomplete dictionary that reconstructs a network's internal activations, have been used to identify these features. However, SAEs may learn more about the structure of the datatset than the computational structure of the network. There is therefore only indirect reason to believe that the directions found in these dictionaries are functionally important to the network. We propose end-to-end (e2e) sparse dictionary learning, a method for training SAEs that ensures the features learned are functionally important by minimizing the KL divergence between the output distributions of the original model and the model with SAE activations inserted. Compared to standard SAEs, e2e SAEs offer a Pareto improvement: They explain more network performance, require fewer total features, and require fewer simultaneously active features per datapoint, all with no cost to interpretability. We explore geometric and qualitative differences between e2e SAE features and standard SAE features. E2e dictionary learning brings us closer to methods that can explain network behavior concisely and accurately. We release our library for training e2e SAEs and reproducing our analysis at https://github.com/ApolloResearch/e2e_sae

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 14 of its 31 references that are in the map. First posted on arXiv 17 May 2024.

Builds on

14 of 31 references.

References · 14

  1. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  2. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  4. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  5. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  6. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
  7. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 171
  8. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  9. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 127
  10. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 106
  11. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 83
  12. 2017SPINE: SParse Interpretable Neural Embeddingscited by 55
2 more
  1. 2023Codebook Features: Sparse and Discrete Interpretability for Neural Networkscited by 45
  2. 2015Sparse Overcomplete Word Vector Representationscited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 27; 2026: 19.

64 papers.

Sort:

By area:

  1. 2024Scaling and evaluating sparse autoencoderscited by 530
  2. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  3. 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 151
  4. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  5. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 109
  6. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 73
  7. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
  8. 2025Weight-sparse transformers have interpretable circuitscited by 40
  9. 2024Efficient Dictionary Learning with Switch Sparse Autoencoderscited by 38
  10. 2024Mathematical Models of Computation in Superpositioncited by 34
  11. 2024Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in GPT-2 Smallcited by 34
  12. 2025Interpreting the linear structure of vision-language model embedding spacescited by 32
38 more
  1. 2024Mechanistic?cited by 25
  2. 2024InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniquescited by 24
  3. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  4. 2024Residual Stream Analysis with Multi-Layer SAEscited by 18
  5. 2024Analyzing (In)Abilities of SAEs via Formal Languagescited by 16
  6. 2024Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Modelscited by 15
  7. 2025I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?cited by 13
  8. 2024Monet: Mixture of Monosemantic Experts for Transformerscited by 12
  9. 2025On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedycited by 12
  10. 2025Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencodercited by 11
  11. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  12. 2025Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistantscited by 10
  13. 2025Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activationscited by 9
  14. 2025HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworkscited by 8
  15. 2025Low-Rank Adapting Models for Sparse Autoencoderscited by 6
  16. 2025Sparse Feature Coactivation Reveals Causal Semantic Modules in Large Language Modelscited by 6
  17. 2025Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretrainingcited by 6
  18. 2025Superposition as Lossy Compression: Measure with Sparse Autoencoders and Connect to Adversarial Vulnerabilitycited by 6
  19. 2024Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learningcited by 5
  20. 2025Scaling sparse feature circuit finding for in-context learningcited by 5
  21. 2026Stable and Steerable Sparse Autoencoders with Weight Regularizationcited by 5
  22. 2024Cluster-norm for Unsupervised Probing of Knowledgecited by 4
  23. 2025Revisiting End-To-End Sparse Autoencoder Training: A Short Finetune Is All You Needcited by 3
  24. 2025Beyond Input Activations: Identifying Influential Latents by Gradient Sparse Autoencoderscited by 3
  25. 2025Evaluating Sparse Autoencoders for Monosemantic Representationcited by 3
  26. 2026Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretabilitycited by 3
  27. 2024Investigating Sensitive Directions in GPT-2: An Improved Baseline and Comparative Analysis of SAEscited by 2
  28. 2024Can Input Attributions Explain Inductive Reasoning in In-Context Learning?cited by 2
  29. 2025FineScope : SAE-guided Data Selection Enables Domain Specific LLM Pruning and Finetuningcited by 2
  30. 2025Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.iicited by 2
  31. 2025Efficient Sketching and Nearest Neighbor Search Algorithms for Sparse Vector Setscited by 1
  32. 2025Beyond Redundancy: Diverse and Specialized Multi-Expert Sparse Autoencodercited by 1
  33. 2026NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agentscited by 1
  34. 2026Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effectscited by 1
  35. 2026When Do Concepts Become Functionally Sufficient During Language-Model Training?cited by 1
  36. 2025Discovering Chunks in Neural Embeddings for Interpretabilitycited by 0
  37. 2025Automated Feature Labeling with Token-Space Gradient Descentcited by 0
  38. 2025Empirical Evaluation of Progressive Coding for Sparse Autoencoderscited by 0

50 most cited of 64 All 64 (JSON)