mlsanityData to 14 Oct 2026

A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders

arXiv 2409.14507 ↗First posted 22 Sep 2024cs.CL cs.AIcited by 160

On the map: Mechanistic interpretability → SAE features and their applications

Abstract (arXiv)

Sparse Autoencoders (SAEs) aim to decompose the activation space of large language models (LLMs) into human-interpretable latent directions or features. As we increase the number of features in the SAE, hierarchical features tend to split into finer features ("math" may split into "algebra", "geometry", etc.), a phenomenon referred to as feature splitting. However, we show that sparse decomposition and splitting of hierarchical features is not robust. Specifically, we show that seemingly monosemantic features fail to fire where they should, and instead get "absorbed" into their children features. We coin this phenomenon feature absorption, and show that it is caused by optimizing for sparsity in SAEs whenever the underlying features form a hierarchy. We introduce a metric to detect absorption in SAEs, and validate our findings empirically on hundreds of LLM SAEs. Our investigation suggests that varying SAE sizes or sparsity is insufficient to solve this issue. We discuss the implications of feature absorption in SAEs and some potential approaches to solve the fundamental theoretical issues before SAEs can be used for interpreting LLMs robustly and at scale.

Key facts. Cited by 160 papers in the map (data through 14 Oct 2026). Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability. Builds on 17 of its 32 references that are in the map. First posted on arXiv 22 Sep 2024.

Builds on

17 of 32 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models31 Jul
  2. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders19 Jul
  3. 2024Scaling and evaluating sparse autoencoders6 Jun
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Models15 Sep

Other references · 13

  1. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  4. 2022Toy Models of Superpositioncited by 971
  5. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  6. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  7. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  8. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  9. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  10. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  11. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  12. 2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43
1 more
  1. 2022What do tokens know about their characters and how do they know it?cited by 36

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 68; 2026: 89.

160 papers.

Sort:

By area:

  1. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 143
  2. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 124
  3. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 115
  4. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 95
  5. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  6. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 73
  7. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  8. 2025Interpreting CLIP with Hierarchical Sparse Autoencoderscited by 60
  9. 2025Detecting High-Stakes Interactions with Activation Probescited by 53
  10. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 49
  11. 2026When Models Manipulate Manifolds: The Geometry of a Counting Taskcited by 46
  12. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 45
38 more
  1. 2025Interpretable Steering of Large Language Models with Feature Guided Activation Additionscited by 35
  2. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  3. 2026Do Sparse Autoencoders Capture Concept Manifolds?cited by 31
  4. 2024Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoderscited by 27
  5. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  6. 2024An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generationcited by 25
  7. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 25
  8. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 24
  9. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 21
  10. 2025Temporal Sparse Autoencoders: Leveraging the Sequential Nature of Language for Interpretabilitycited by 18
  11. 2025Dense SAE Latents Are Features, Not Bugscited by 17
  12. 2025Feature Hedging: Correlated Features Break Narrow Sparse Autoencoderscited by 16
  13. 2025Stochastic Parameter Decompositioncited by 16
  14. 2025Feature-Level Insights into Artificial Text Detection with Sparse Autoencoderscited by 15
  15. 2025Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEscited by 14
  16. 2025Incorporating Hierarchical Semantics in Sparse Autoencoder Architecturescited by 14
  17. 2025Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoderscited by 14
  18. 2025Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoderscited by 13
  19. 2025Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoderscited by 12
  20. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?cited by 12
  21. 2026From Directions to Regions: Decomposing Activations in Language Models via Local Geometrycited by 12
  22. 2025Explainability in Practice: A Survey of Explainable NLP Across Various Domainscited by 11
  23. 2025Understanding sparse autoencoder scaling in the presence of feature manifoldscited by 11
  24. 2026SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Datacited by 11
  25. 2025Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activationscited by 10
  26. 2025Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoderscited by 9
  27. 2025Denoising Concept Vectors with Sparse Autoencoders for Improved Language Model Steeringcited by 9
  28. 2025AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Featurescited by 9
  29. 2025Do Sparse Autoencoders Generalize? A Case Study of Answerabilitycited by 8
  30. 2025Mechanistic Indicators of Understanding in Large Language Modelscited by 8
  31. 2025OrtSAE: Orthogonal Sparse Autoencoders Uncover Atomic Featurescited by 8
  32. 2025Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scalingcited by 8
  33. 2026AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoderscited by 8
  34. 2025Low-Rank Adapting Models for Sparse Autoencoderscited by 7
  35. 2025Superposition as Lossy Compression: Measure with Sparse Autoencoders and Connect to Adversarial Vulnerabilitycited by 7
  36. 2025LLM Interpretability with Identifiable Temporal-Instantaneous Representationcited by 6
  37. 2025Interpretable and Steerable Concept Bottleneck Sparse Autoencoderscited by 6
  38. 2026From Atoms to Trees: Building a Structured Feature Forest with Hierarchical Sparse Autoencoderscited by 6

50 most cited of 160 All 160 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models
  2. 2025Sparse Autoencoders Trained on the Same Data Learn Different Features
  3. 2025On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedy
  4. 2025Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations
  5. 2024Automatically Interpreting Millions of Features in Large Language Models
  6. 2025Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages
  7. 2026How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models
  8. 2025Evaluating Sparse Autoencoders for Monosemantic Representation