mlsanityData to 14 Oct 2026

Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs

arXiv 2505.20254 ↗First posted 26 May 2025cs.LG cs.AI cs.CLcited by 14

On the map: Mechanistic interpretability → Sparse autoencoder training and evaluation

Abstract (arXiv)

Sparse Autoencoders (SAEs) are a prominent tool in mechanistic interpretability (MI) for decomposing neural network activations into interpretable features. However, the aspiration to identify a canonical set of features is challenged by the observed inconsistency of learned SAE features across different training runs, undermining the reliability and efficiency of MI research. This position paper argues that mechanistic interpretability should prioritize feature consistency in SAEs -- the reliable convergence to equivalent feature sets across independent runs. We propose using the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC) as a practical metric to operationalize consistency and demonstrate that high levels are achievable (0.80 for TopK SAEs on LLM activations) with appropriate architectural choices. Our contributions include detailing the benefits of prioritizing consistency; providing theoretical grounding and synthetic validation using a model organism, which verifies PW-MCC as a reliable proxy for ground-truth recovery; and extending these findings to real-world LLM data, where high feature consistency strongly correlates with the semantic similarity of learned feature explanations. We call for a community-wide shift towards systematically measuring feature consistency to foster robust cumulative progress in MI.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability. Builds on 32 of its 45 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

32 of 45 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoders21 Mar
  2. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry3 Mar
  3. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models18 Feb
  4. 2025Sparse Autoencoders Do Not Find Canonical Units of Analysis7 Feb
  5. 2024Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders20 Nov
  6. 2024Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoders2 Nov
  7. 2022Toy Models of Superposition21 Sep

Other references · 25

  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  2. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  4. 2024Scaling and evaluating sparse autoencoderscited by 553
  5. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  6. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
  7. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  8. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 179
  9. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
  10. 2024BatchTopK Sparse Autoencoderscited by 130
  11. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 124
  12. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 113
13 more
  1. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 95
  2. 2025Sparse Autoencoders Trained on the Same Data Learn Different Featurescited by 93
  3. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 92
  4. 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patchingcited by 80
  5. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 71
  6. 2024Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Modelscited by 51
  7. 2025Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?cited by 34
  8. 2013New Algorithms for Learning Incoherent and Overcomplete Dictionariescited by 22
  9. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMscited by 22
  10. 2022Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomenacited by 21
  11. 2025Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activationscited by 9
  12. 2025SAFE: A Sparse Autoencoder-Based Framework for Robust Query Enrichment and Hallucination Mitigation in LLMscited by 8
  13. 2024SCAR: Sparse Conditioned Autoencoders for Concept Detection and Steering in LLMscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 10.

14 papers.

Sort:

By area:

  1. 2026SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Datacited by 11
  2. 2025Attribution-Guided Distillation of Matryoshka Sparse Autoencoderscited by 4
  3. 2026Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretabilitycited by 4
  4. 2026Are Sparse Autoencoder Benchmarks Reliable?cited by 4
  5. 2025Interpretability as Alignment: Making Internal Understanding a Design Principlecited by 3
  6. 2026Toward Identifiable Sparse Autoencoderscited by 2
  7. 2026Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewingcited by 2
  8. 2025Enforcing Orderedness to Improve Feature Consistencycited by 1
  9. 2025Graph-Regularized Sparse Autoencoders for LLM Safety Steeringcited by 1
  10. 2026A Unifying Framework for Unsupervised Concept Extractioncited by 1
  11. 2026Bilinear autoencoders find interpretable manifoldscited by 1
  12. 2026Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)cited by 1
2 more
  1. 2026Who Guards the Guardians? The Challenges of Evaluating Identifiability of Learned Representationscited by 0
  2. 2026Where Decoder Cosine Similarity Fails for SAE Feature Flow Discoverycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers
  2. 2026Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?
  3. 2025Evaluating SAE interpretability without explanations
  4. 2024Mechanistic Permutability: Match Features Across Layers
  5. 2025Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data
  6. 2026Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility
  7. 2024Interpreting Attention Layer Outputs with Sparse Autoencoders
  8. 2024Evaluating Sparse Autoencoders on Targeted Concept Erasure Tasks