mlsanityData to 7 Oct 2026

k-Sparse Autoencoders

arXiv 1312.5663 ↗First posted 19 Dec 2013cs.LGcited by 209

Not on the map. Why?

Abstract (arXiv)

Recently, it has been observed that when representations are learnt in a way that encourages sparsity, improved performance is obtained on classification tasks. These methods involve combinations of activation functions, sampling steps and different kinds of penalties. To investigate the effectiveness of sparsity by itself, we propose the k-sparse autoencoder, which is an autoencoder with linear activation function, where in hidden layers only the k highest activities are kept. When applied to the MNIST and NORB datasets, we find that this method achieves better classification results than denoising autoencoders, networks trained with dropout, and RBMs. k-sparse autoencoders are simple to train and the encoding stage is very fast, making them well-suited to large problem sizes, where conventional sparse coding algorithms cannot be applied.

Key facts. Cited by 209 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Dec 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 7; 2023: 10; 2024: 27; 2025: 77; 2026: 74.

209 papers.

Sort:

By area:

  1. 2024Scaling and evaluating sparse autoencoderscited by 530
  2. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  3. 2023A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimationcited by 108
  4. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  5. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 75
  6. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 70
  7. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 58
  8. 2024Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Modelscited by 51
  9. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 49
  10. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  11. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 42
  12. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 39
38 more
  1. 2024Efficient Dictionary Learning with Switch Sparse Autoencoderscited by 38
  2. 2025LLM Pretraining with Continuous Conceptscited by 33
  3. 2025Interpreting the linear structure of vision-language model embedding spacescited by 32
  4. 2024textitRevelio: Interpreting and leveraging semantic information in diffusion modelscited by 29
  5. 2024Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoderscited by 25
  6. 2024Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoderscited by 25
  7. 2025Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representationcited by 25
  8. 2025Route Sparse Autoencoder to Interpret Large Language Modelscited by 25
  9. 2025Base Models Know How to Reason, Thinking Models Learn Whencited by 25
  10. 2025Sparse Autoencoders for Hypothesis Generationcited by 24
  11. 2025Interpretable and Testable Vision Features via Sparse Autoencoderscited by 24
  12. 2023Geometric Autoencoders -- What You See is What You Decodecited by 23
  13. 2021Regularising Inverse Problems with Generative Machine Learning Modelscited by 22
  14. 2024Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEscited by 22
  15. 2025Mixture of Experts Made Intrinsically Interpretablecited by 22
  16. 2025Interpreting and Steering LLMs with Mutual Information-based Explanations on Sparse Autoencoderscited by 21
  17. 2025Emergence and Evolution of Interpretable Concepts in Diffusion Modelscited by 20
  18. 2023Discovering Universal Geometry in Embeddings with ICAcited by 19
  19. 2024Disentangling Dense Embeddings with Sparse Autoencoderscited by 19
  20. 2022Sparsity and Heterogeneous Dropout for Continual Learning in the Null Space of Neural Activationscited by 18
  21. 2024Residual Stream Analysis with Multi-Layer SAEscited by 18
  22. 2021Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularizationcited by 17
  23. 2024Concept Bottleneck Models Without Predefined Conceptscited by 17
  24. 2024Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoderscited by 17
  25. 2024Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionariescited by 17
  26. 2025Position: Use Sparse Autoencoders to Discover Unknownscited by 17
  27. 2025TIDE : Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generationcited by 16
  28. 2021Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networkscited by 15
  29. 2022On the link between conscious function and general intelligence in humans and machinescited by 15
  30. 2025Analyze Feature Flow to Enhance Interpretation and Steering in Language Modelscited by 12
  31. 2025On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedycited by 12
  32. 2025Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencodercited by 11
  33. 2025Discovering and Steering Interpretable Concepts in Large Generative Music Modelscited by 11
  34. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  35. 2024Adaptive Sparse Allocation with Mutual Choice & Feature Choice Sparse Autoencoderscited by 9
  36. 2025ConSim: Measuring Concept-Based Explanations' Effectiveness with Automated Simulatabilitycited by 9
  37. 2026Cross-Modal Redundancy and the Geometry of Vision-Language Embeddingscited by 9
  38. 2025Sparse Autoencoder as a Zero-Shot Classifier for Concept Erasing in Text-to-Image Diffusion Modelscited by 8

50 most cited of 209 All 209 (JSON)