# On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoders

arXiv: https://arxiv.org/abs/2605.31518 · HTML: https://mlsanity.com/paper/2605.31518/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability.
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 29 May 2026.

## Placed in

- Mechanistic interpretability / [Sparse autoencoder training and evaluation](https://mlsanity.com/mechinterp/sparse-autoencoder-training-and-evaluation/)

## Builds on (most cited first, 24 of 24)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders](https://mlsanity.com/paper/2407.14435/) (2024)
- [Massive Activations in Large Language Models](https://mlsanity.com/paper/2402.17762/) (2024)
- [Transcoders Find Interpretable LLM Feature Circuits](https://mlsanity.com/paper/2406.11944/) (2024)
- [Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet](https://mlsanity.com/paper/2605.29358/) (2026)
- [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022)
- [Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing](https://mlsanity.com/paper/2306.12929/) (2023)
- [Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models](https://mlsanity.com/paper/2504.02821/) (2025)
- [InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders](https://mlsanity.com/paper/2412.12101/) (2024)
- [Outlier-Efficient Hopfield Layers for Large Transformer-Based Models](https://mlsanity.com/paper/2404.03828/) (2024)
- [Steering CLIP's vision transformer with sparse autoencoders](https://mlsanity.com/paper/2504.08729/) (2025)
- [The Missing Curve Detectors of InceptionV1: Applying Sparse Autoencoders to InceptionV1 Early Vision](https://mlsanity.com/paper/2406.03662/) (2024)
- [A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training](https://mlsanity.com/paper/2601.22966/) (2026)
- [Fast and Low-Cost Genomic Foundation Models via Outlier Removal](https://mlsanity.com/paper/2505.00598/) (2025)
- [Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning](https://mlsanity.com/paper/2508.16929/) (2025)
- [Data Whitening Improves Sparse Autoencoder Learning](https://mlsanity.com/paper/2511.13981/) (2025)

## Built on by (most cited first, 2 of 2)

- [Transformer MLP Gate Thresholds Are Couplings to a Carried Reference Direction](https://mlsanity.com/paper/2609.31956/) (2026)
- [D-Scope: Decomposing and Steering Diffusion Transformers with Sparse Autoencoders](https://mlsanity.com/paper/2609.39625/) (2026)

## Abstract (arXiv)

Sparse autoencoders (SAEs) decompose neural network activations into interpretable features, but many learned features never activate, a problem called feature death that wastes dictionary capacity and can reintroduce superposition. Death rates vary dramatically between models: near-zero on GPT-2, over 70% on AlphaFold3 with identical configurations. We find that dimension-level activation outliers (dimensions whose mean magnitude is large relative to per-token variation) cause this by shifting pre-activations at initialization based on each feature's alignment with the activation mean. Features anti-aligned with the mean receive permanently negative pre-activations and never fire. We formalize outlier severity as γ = |μ|/|σ|; it predicts initial death rates (Spearman ρ = 0.89 for dead-by-TopK, 0.82 for dead-by-ReLU) across 454 model-layer combinations spanning language, vision, protein, and genomic models. Dead features can revive during training, but recovery requires the SAE bias to learn the activation mean, a process that is prohibitively slow at high γ. Mean-centering (subtracting the activation mean) sidesteps this and eliminates outlier-induced death across all tested models, confirming the mechanism and providing a principled basis for when and why this preprocessing step is necessary.