mlsanityData to 7 Oct 2026

Evolution of SAE Features Across Layers in LLMs

arXiv 2410.08869 ↗First posted 11 Oct 2024cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Sparse Autoencoders for transformer-based language models are typically defined independently per layer. In this work we analyze statistical relationships between features in adjacent layers to understand how features evolve through a forward pass. We provide a graph visualization interface for features and their most similar next-layer neighbors (https://stefanhex.com/spar-2024/feature-browser/), and build communities of related features across layers. We find that a considerable amount of features are passed through from a previous layer, some features can be expressed as quasi-boolean combinations of previous features, and some features become more specialized in later layers.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 6 of its 13 references that are in the map. First posted on arXiv 11 Oct 2024.

Builds on

6 of 13 references.

References · 6

  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  3. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  4. 2018From Louvain to Leiden: guaranteeing well-connected communitiescited by 379
  5. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  6. 2024Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networkscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 5.

14 papers.

Sort:

  1. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 39
  2. 2025Analyze Feature Flow to Enhance Interpretation and Steering in Language Modelscited by 12
  3. 2025Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activationscited by 9
  4. 2025How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoderscited by 5
  5. 2025Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Conceptscited by 2
  6. 2025Automated Attribution Graph Interpretation via Probe Promptingcited by 2
  7. 2025DePass: Unified Feature Attributing by Simple Decomposed Forward Passcited by 1
  8. 2026Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effectscited by 1
  9. 2025SCALAR: Benchmarking SAE Interaction Sparsity in Toy LLMscited by 0
  10. 2025Uncovering Competency Gaps in Large Language Models and Their Benchmarkscited by 0
  11. 2026Domain-Filtered Knowledge Graphs from Sparse Autoencoder Featurescited by 0
  12. 2026Unstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse Autoencoderscited by 0
2 more
  1. 2026Why Knowing Both Hops Is Not Enough: Understanding Two-Hop Generalization in Language Modelscited by 0
  2. 2026Where Decoder Cosine Similarity Fails for SAE Feature Flow Discoverycited by 0