# Automatically Interpreting Millions of Features in Large Language Models

arXiv: https://arxiv.org/abs/2410.13928 · HTML: https://mlsanity.com/paper/2410.13928/

## Key facts

- Cited by 113 papers in the map (data through 14 Oct 2026).
- Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability.
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Placed in

- Mechanistic interpretability / [SAE features and their applications](https://mlsanity.com/mechinterp/sae-features-and-their-applications/)

## Builds on (most cited first, 22 of 22)

- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [Git Re-Basin: Merging Models modulo Permutation Symmetries](https://mlsanity.com/paper/2209.04836/) (2022)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models](https://mlsanity.com/paper/2401.06102/) (2024)
- [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022)
- [Linear Algebraic Structure of Word Senses, with Applications to Polysemy](https://mlsanity.com/paper/1601.03764/) (2016)
- [Invariance, Causality and Robustness](https://mlsanity.com/paper/1812.08233/) (2018)
- [Universal Neurons in GPT2 Language Models](https://mlsanity.com/paper/2401.12181/) (2024)
- [SelfIE: Self-Interpretation of Large Language Model Embeddings](https://mlsanity.com/paper/2403.10949/) (2024)
- [A Multimodal Automated Interpretability Agent](https://mlsanity.com/paper/2404.14394/) (2024)
- [Explaining black box text modules in natural language with language models](https://mlsanity.com/paper/2305.09863/) (2023)
- [Rigorously Assessing Natural Language Explanations of Neurons](https://mlsanity.com/paper/2309.10312/) (2023)
- [CoSy: Evaluating Textual Explanations of Neurons](https://mlsanity.com/paper/2405.20331/) (2024)
- [The Importance of Prompt Tuning for Automated Neuron Explanations](https://mlsanity.com/paper/2310.06200/) (2023)

## Built on by (most cited first, 30 of 113)

- [Learning Multi-Level Features with Matryoshka Sparse Autoencoders](https://mlsanity.com/paper/2503.17547/) (2025)
- [Steering Language Model Refusal with Sparse Autoencoders](https://mlsanity.com/paper/2411.11296/) (2024)
- [SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders](https://mlsanity.com/paper/2501.18052/) (2025)
- [Verbalizable Representations Form a Global Workspace in Language Models](https://mlsanity.com/paper/2607.15495/) (2026)
- [Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers](https://mlsanity.com/paper/2512.15674/) (2025)
- [When Models Manipulate Manifolds: The Geometry of a Counting Task](https://mlsanity.com/paper/2601.04480/) (2026)
- [I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders](https://mlsanity.com/paper/2503.18878/) (2025)
- [Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning](https://mlsanity.com/paper/2507.16795/) (2025)
- [Interpretable Steering of Large Language Models with Feature Guided Activation Additions](https://mlsanity.com/paper/2501.09929/) (2025)
- [Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning](https://mlsanity.com/paper/2504.02922/) (2025)
- [Training Language Models to Explain Their Own Computations](https://mlsanity.com/paper/2511.08579/) (2025)
- [Mixture of Experts Made Intrinsically Interpretable](https://mlsanity.com/paper/2503.07639/) (2025)
- [Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models](https://mlsanity.com/paper/2605.11887/) (2026)
- [Dense SAE Latents Are Features, Not Bugs](https://mlsanity.com/paper/2506.15679/) (2025)
- [Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs](https://mlsanity.com/paper/2505.20254/) (2025)
- [On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedy](https://mlsanity.com/paper/2506.15963/) (2025)
- [Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?](https://mlsanity.com/paper/2602.14111/) (2026)
- [Shaping capabilities with token-level data filtering](https://mlsanity.com/paper/2601.21571/) (2026)
- [Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencoder](https://mlsanity.com/paper/2502.14050/) (2025)
- [From Directions to Regions: Decomposing Activations in Language Models via Local Geometry](https://mlsanity.com/paper/2602.02464/) (2026)
- [SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data](https://mlsanity.com/paper/2602.14687/) (2026)
- [Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations](https://mlsanity.com/paper/2502.18147/) (2025)
- [Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework](https://mlsanity.com/paper/2506.15538/) (2025)
- [Measuring and Guiding Monosemanticity](https://mlsanity.com/paper/2506.19382/) (2025)
- [Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoders](https://mlsanity.com/paper/2510.03659/) (2025)
- [Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders](https://mlsanity.com/paper/2507.23220/) (2025)
- [OrtSAE: Orthogonal Sparse Autoencoders Uncover Atomic Features](https://mlsanity.com/paper/2509.22033/) (2025)
- [How and Why LLMs Generalize: A Fine-Grained Analysis of LLM Reasoning from Cognitive Behaviors to Low-Level Patterns](https://mlsanity.com/paper/2512.24063/) (2025)
- [Causality is Key for Interpretability Claims to Generalise](https://mlsanity.com/paper/2602.16698/) (2026)
- [Circuit Tracing in Vision-Language Models: Understanding the Internal Mechanisms of Multimodal Thinking](https://mlsanity.com/paper/2602.20330/) (2026)

## Abstract (arXiv)

While the activations of neurons in deep neural networks usually do not have a simple human-understandable interpretation, sparse autoencoders (SAEs) can be used to transform these activations into a higher-dimensional latent space which may be more easily interpretable. However, these SAEs can have millions of distinct latent features, making it infeasible for humans to manually interpret each one. In this work, we build an open-source automated pipeline to generate and evaluate natural language explanations for SAE features using LLMs. We test our framework on SAEs of varying sizes, activation functions, and losses, trained on two different open-weight LLMs. We introduce five new techniques to score the quality of explanations that are cheaper to run than the previous state of the art. One of these techniques, intervention scoring, evaluates the interpretability of the effects of intervening on a feature, which we find explains features that are not recalled by existing methods. We propose guidelines for generating better explanations that remain valid for a broader set of activating contexts, and discuss pitfalls with existing scoring techniques. We use our explanations to measure the semantic similarity of independently trained SAEs, and find that SAEs trained on nearby layers of the residual stream are highly similar. Our large-scale analysis confirms that SAE latents are indeed much more interpretable than neurons, even when neurons are sparsified using top-k postprocessing. Our code is available at https://github.com/EleutherAI/sae-auto-interp, and our explanations are available at https://huggingface.co/datasets/EleutherAI/autointerpexplanations.