Data to 7 Oct 2026

Sparse Autoencoders Trained on the Same Data Learn Different Features

arXiv 2501.16615 ↗First posted 28 Jan 2025cs.LGcited by 86

Not on the map. Why?

Abstract (arXiv)

Sparse autoencoders (SAEs) are a useful tool for uncovering human-interpretable features in the activations of large language models (LLMs). While some expect SAEs to find the true underlying features used by a model, our research shows that SAEs trained on the same model and data, differing only in the random seed used to initialize their weights, identify different sets of features. For example, in an SAE with 131K latents trained on a feedforward network in Llama 3 8B, only 30% of the features were shared across different seeds. We observed this phenomenon across multiple layers of three different LLMs, two datasets, and several SAE architectures. While ReLU SAEs trained with the L1 sparsity loss showed greater stability across seeds, SAEs using the state-of-the-art TopK activation function were more seed-dependent, even when controlling for the level of sparsity. Our results suggest that the set of features uncovered by an SAE should be viewed as a pragmatically useful decomposition of activation space, rather than an exhaustive and universal list of features "truly used" by the model.

Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Jan 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 40; 2026: 46.

86 papers.

Sort:

By area:

2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 58
2025Interpreting CLIP with Hierarchical Sparse Autoencoderscited by 54
2025Interpreting the linear structure of vision-language model embedding spacescited by 32
2025Base Models Know How to Reason, Thinking Models Learn Whencited by 25
2026Do Sparse Autoencoders Capture Concept Manifolds?cited by 25
2025Transferring Linear Features Across Language Models With Model Stitchingcited by 17
2025From superposition to sparse codes: interpretable representations in neural networkscited by 16
2025Temporal Sparse Autoencoders: Leveraging the Sequential Nature of Language for Interpretabilitycited by 16
2025Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEscited by 14
2025Towards Interpretable Protein Structure Prediction with Sparse Autoencoderscited by 12
2026How Many Features Can a Language Model Store Under the Linear Representation Hypothesis?cited by 11

38 more

2025Measuring and Guiding Monosemanticitycited by 10
2026Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?cited by 10
2025Can Interpretation Predict Behavior on Unseen Data?cited by 8
2026The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Levelcited by 7
2025Evaluating and Designing Sparse Autoencoders by Approximating Quasi-Orthogonalitycited by 6
2025Disentangling Polysemantic Channels in Convolutional Neural Networkscited by 6
2025Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attributioncited by 5
2025Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoderscited by 5
2025Finding Manifolds With Bilinear Autoencoderscited by 5
2026Stable and Steerable Sparse Autoencoders with Weight Regularizationcited by 5
2026Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisationcited by 5
2025Feature Identification via the Empirical NTKcited by 4
2025Superposition disentanglement of neural representations reveals hidden alignmentcited by 4
2025Attribution-Guided Distillation of Matryoshka Sparse Autoencoderscited by 4
2026Operationalising the Superficial Alignment Hypothesis via Task Complexitycited by 4
2026Steering at the Source: Style Modulation Heads for Robust Persona Controlcited by 4
2026Perplexity Can Miss SAE Feature Damage Under Quantizationcited by 4
2025Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influencecited by 3
2025Ensembling Sparse Autoencoderscited by 3
2025A Geometric Unification of Concept Learning with Concept Conescited by 3
2026Towards Worst-Case Guarantees with Scale-Aware Interpretabilitycited by 3
2026Are Sparse Autoencoder Benchmarks Reliable?cited by 3
2025Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoringcited by 2
2025Prompting as Scientific Inquirycited by 2
2025Resurrecting the Salmon: Rethinking Mechanistic Interpretability with Domain-Specific Sparse Autoencoderscited by 2
2025Discovering Interpretable Biological Concepts in Single-cell RNA-seq Foundation Modelscited by 2
2025Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Conceptscited by 2
2025Circuits, Features, and Heuristics in Molecular Transformerscited by 2
2026Superposition Without Interference? Towards Isolated Interventions via Almost Orthogonal Features in Language Modelscited by 2
2026Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Featurescited by 2
2026Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancycited by 2
2026Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselinescited by 2
2026Toward Identifiable Sparse Autoencoderscited by 2
2026Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamicscited by 2
2025Cross-Layer Discrete Concept Discovery for Interpreting Language Modelscited by 1
2025A Review of Developmental Interpretability in Large Language Modelscited by 1
2025Towards Open-Ended Visual Scientific Discovery with Sparse Autoencoderscited by 1
2025Graph-Regularized Sparse Autoencoders for LLM Safety Steeringcited by 1

50 most cited of 86 All 86 (JSON)