mlsanityData to 7 Oct 2026

Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders

arXiv 2407.14435 ↗First posted 19 Jul 2024cs.LGcited by 283

Not on the map. Why?

Abstract (arXiv)

Sparse autoencoders (SAEs) are a promising unsupervised approach for identifying causally relevant and interpretable linear features in a language model's (LM) activations. To be useful for downstream tasks, SAEs need to decompose LM activations faithfully; yet to be interpretable the decomposition must be sparse -- two objectives that are in tension. In this paper, we introduce JumpReLU SAEs, which achieve state-of-the-art reconstruction fidelity at a given sparsity level on Gemma 2 9B activations, compared to other recent advances such as Gated and TopK SAEs. We also show that this improvement does not come at the cost of interpretability through manual and automated interpretability studies. JumpReLU SAEs are a simple modification of vanilla (ReLU) SAEs -- where we replace the ReLU with a discontinuous JumpReLU activation function -- and are similarly efficient to train and run. By utilising straight-through-estimators (STEs) in a principled manner, we show how it is possible to train JumpReLU SAEs effectively despite the discontinuous JumpReLU function introduced in the SAE's forward pass. Similarly, we use STEs to directly train L0 to be sparse, instead of training on proxies such as L1, avoiding problems like shrinkage.

Key facts. Cited by 283 papers in the map (data through 7 Oct 2026). Builds on 15 of its 40 references that are in the map. First posted on arXiv 19 Jul 2024.

Builds on

15 of 40 references.

References · 15

  1. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  2. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  4. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  6. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 571
  7. 2024Scaling and evaluating sparse autoencoderscited by 530
  8. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  9. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  10. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  11. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
  12. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
3 more
  1. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 171
  2. 2014Zero-bias autoencoders and the benefits of co-adapting featurescited by 15
  3. 2019JumpReLU: A Retrofit Defense Strategy for Adversarial Attackscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 29; 2025: 126; 2026: 128.

283 papers.

Sort:

By area:

  1. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  2. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  3. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 136
  4. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 133
  5. 2024BatchTopK Sparse Autoencoderscited by 126
  6. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 118
  7. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  8. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 109
  9. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  10. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 86
  11. 2025Persona Features Control Emergent Misalignmentcited by 76
  12. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 74
38 more
  1. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 73
  2. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 70
  3. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
  4. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  5. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 60
  6. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 58
  7. 2025Interpreting CLIP with Hierarchical Sparse Autoencoderscited by 54
  8. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  9. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  10. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 42
  11. 2025Weight-sparse transformers have interpretable circuitscited by 40
  12. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 39
  13. 2024Efficient Dictionary Learning with Switch Sparse Autoencoderscited by 38
  14. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  15. 2025SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Modelscited by 32
  16. 2025Interpreting the linear structure of vision-language model embedding spacescited by 32
  17. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 31
  18. 2024Towards Unifying Interpretability and Control: Evaluation via Interventioncited by 28
  19. 2025Towards Understanding Distilled Reasoning Models: A Representational Approachcited by 26
  20. 2024Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoderscited by 25
  21. 2024Mechanistic?cited by 25
  22. 2024Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoderscited by 25
  23. 2025Route Sparse Autoencoder to Interpret Large Language Modelscited by 25
  24. 2026Do Sparse Autoencoders Capture Concept Manifolds?cited by 25
  25. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  26. 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 22
  27. 2024On the Complexity of Neural Computation in Superpositioncited by 21
  28. 2024Mechanistic Permutability: Match Features Across Layerscited by 21
  29. 2025Interpreting and Steering LLMs with Mutual Information-based Explanations on Sparse Autoencoderscited by 21
  30. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMscited by 21
  31. 2025Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Videocited by 20
  32. 2024Can sparse autoencoders be used to decompose and interpret steering vectors?cited by 19
  33. 2024Residual Stream Analysis with Multi-Layer SAEscited by 18
  34. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 18
  35. 2024Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoderscited by 17
  36. 2025Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Wordscited by 17
  37. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projectionscited by 17
  38. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 17

50 most cited of 283 All 283 (JSON)