# Evolution of SAE Features Across Layers in LLMs

arXiv: https://arxiv.org/abs/2410.08869 · HTML: https://mlsanity.com/paper/2410.08869/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability.
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Placed in

- Mechanistic interpretability / [SAE features and their applications](https://mlsanity.com/mechinterp/sae-features-and-their-applications/)

## Builds on (most cited first, 6 of 6)

- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [From Louvain to Leiden: guaranteeing well-connected communities](https://mlsanity.com/paper/1810.08473/) (2018)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networks](https://mlsanity.com/paper/2407.04690/) (2024)

## Built on by (most cited first, 14 of 14)

- [Enhancing Automated Interpretability with Output-Centric Feature Descriptions](https://mlsanity.com/paper/2501.08319/) (2025)
- [Analyze Feature Flow to Enhance Interpretation and Steering in Language Models](https://mlsanity.com/paper/2502.03032/) (2025)
- [Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations](https://mlsanity.com/paper/2502.18147/) (2025)
- [How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders](https://mlsanity.com/paper/2503.06394/) (2025)
- [Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Concepts](https://mlsanity.com/paper/2511.06048/) (2025)
- [Automated Attribution Graph Interpretation via Probe Prompting](https://mlsanity.com/paper/2511.07002/) (2025)
- [DePass: Unified Feature Attributing by Simple Decomposed Forward Pass](https://mlsanity.com/paper/2510.18462/) (2025)
- [Unstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse Autoencoders](https://mlsanity.com/paper/2606.12138/) (2026)
- [Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects](https://mlsanity.com/paper/2607.20596/) (2026)
- [SCALAR: Benchmarking SAE Interaction Sparsity in Toy LLMs](https://mlsanity.com/paper/2511.07572/) (2025)
- [Uncovering Competency Gaps in Large Language Models and Their Benchmarks](https://mlsanity.com/paper/2512.20638/) (2025)
- [Domain-Filtered Knowledge Graphs from Sparse Autoencoder Features](https://mlsanity.com/paper/2604.23829/) (2026)
- [Why Knowing Both Hops Is Not Enough: Understanding Two-Hop Generalization in Language Models](https://mlsanity.com/paper/2608.07261/) (2026)
- [Where Decoder Cosine Similarity Fails for SAE Feature Flow Discovery](https://mlsanity.com/paper/2609.12591/) (2026)

## Abstract (arXiv)

Sparse Autoencoders for transformer-based language models are typically defined independently per layer. In this work we analyze statistical relationships between features in adjacent layers to understand how features evolve through a forward pass. We provide a graph visualization interface for features and their most similar next-layer neighbors (https://stefanhex.com/spar-2024/feature-browser/), and build communities of related features across layers. We find that a considerable amount of features are passed through from a previous layer, some features can be expressed as quasi-boolean combinations of previous features, and some features become more specialized in later layers.