mlsanityData to 14 Oct 2026

Sparse Feature Coactivation Reveals Causal Semantic Modules in Large Language Models

arXiv 2506.18141 ↗First posted 22 Jun 2025cs.CL cs.AIcited by 6

On the map: Mechanistic interpretability → SAE features and their applications

Abstract (arXiv)

We identify semantically coherent, context-consistent network components in large language models (LLMs) using coactivation of sparse autoencoder (SAE) features collected from just a handful of prompts. Focusing on concept-relation prediction tasks, we show that ablating these components for concepts (e.g., countries and words) and relations (e.g., capital city and translation language) changes model outputs in predictable ways, while amplifying these components induces counterfactual responses. Notably, composing relation and concept components yields compound counterfactual outputs. Further analysis reveals that while most concept components emerge from the very first layer, more abstract relation components are concentrated in later layers. Lastly, we show that extracted components more comprehensively capture concepts and relations than individual features while maintaining specificity. Overall, our findings suggest a modular organization of knowledge and advance methods for efficient, targeted LLM manipulation.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability. Builds on 28 of its 41 references that are in the map. First posted on arXiv 22 Jun 2025.

Builds on

28 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders19 Jul
  2. 2024Improving Dictionary Learning with Gated Sparse Autoencoders24 Apr
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Models15 Sep

Other references · 25

  1. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  6. 2022In-context Learning and Induction Headscited by 964
  7. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  8. 2021Editing Factual Knowledge in Language Modelscited by 553
  9. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  10. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  11. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  12. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
13 more
  1. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 200
  2. 2023Localizing Model Behavior with Path Patchingcited by 195
  3. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 98
  4. 2024Identifying Functionally Important Features with End-to-End Sparse Dictionary Learningcited by 67
  5. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  6. 2024Finding Transformer Circuits with Edge Pruningcited by 60
  7. 2025Steering Large Language Model Activations in Sparse Spacescited by 58
  8. 2024pyvene: A Library for Understanding and Improving PyTorch Models via Interventionscited by 54
  9. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  10. 2024Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformerscited by 44
  11. 2024The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Unitscited by 34
  12. 2024Efficient Automated Circuit Discovery in Transformers using Contextual Decompositioncited by 23
  13. 2025How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Trainingcited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2026Minimizing Collateral Damage in Activation Steeringcited by 5
  2. 2026From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoningcited by 5
  3. 2025Emergence and Localisation of Semantic Role Circuits in LLMscited by 2
  4. 2025Conflict Adaptation in Vision-Language Modelscited by 1
  5. 2026From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Modelscited by 0
  6. 2026Interference Beyond Geometry in Concept Extractioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models
  2. 2026Do Language Models Encode Semantic Relations? Probing and Sparse Feature Analysis
  3. 2026Concept Component Analysis: A Principled Approach for Concept Extraction in LLMs
  4. 2026From Atoms to Trees: Building a Structured Feature Forest with Hierarchical Sparse Autoencoders
  5. 2025Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations
  6. 2025Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages
  7. 2025Interpreting Language Models Through Concept Descriptions: A Survey
  8. 2025Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoders