# Weights Read and Write Features: Scalable Parameter Decomposition Grounded in Activation Space

arXiv: https://arxiv.org/abs/2609.37731 · HTML: https://mlsanity.com/paper/2609.37731/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 47 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 30 of 37)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024)
- [Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models](https://mlsanity.com/paper/2401.06102/) (2024)
- [Transcoders Find Interpretable LLM Feature Circuits](https://mlsanity.com/paper/2406.11944/) (2024)
- [Open Problems in Mechanistic Interpretability](https://mlsanity.com/paper/2501.16496/) (2025)
- [A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders](https://mlsanity.com/paper/2409.14507/) (2024)
- [Learning Multi-Level Features with Matryoshka Sparse Autoencoders](https://mlsanity.com/paper/2503.17547/) (2025)
- [BatchTopK Sparse Autoencoders](https://mlsanity.com/paper/2412.06410/) (2024)
- [The Geometry of Concepts: Sparse Autoencoder Feature Structure](https://mlsanity.com/paper/2410.19750/) (2024)
- [Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers](https://mlsanity.com/paper/2512.15674/) (2025)
- [Weight-sparse transformers have interpretable circuits](https://mlsanity.com/paper/2511.13653/) (2025)
- [Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks](https://mlsanity.com/paper/2311.09060/) (2023)
- [Bilinear MLPs enable weight-based mechanistic interpretability](https://mlsanity.com/paper/2410.08417/) (2024)
- [Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition](https://mlsanity.com/paper/2501.14926/) (2025)
- [Steering Language Models with Weight Arithmetic](https://mlsanity.com/paper/2511.05408/) (2025)
- [ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models](https://mlsanity.com/paper/2503.22048/) (2025)
- [Stochastic Parameter Decomposition](https://mlsanity.com/paper/2506.20790/) (2025)
- [Analyze Feature Flow to Enhance Interpretation and Steering in Language Models](https://mlsanity.com/paper/2502.03032/) (2025)
- [Precise In-Parameter Concept Erasure in Large Language Models](https://mlsanity.com/paper/2505.22586/) (2025)
- [Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition](https://mlsanity.com/paper/2504.20938/) (2025)
- [Low-Rank Adapting Models for Sparse Autoencoders](https://mlsanity.com/paper/2501.19406/) (2025)

## Abstract (arXiv)

Activation space and parameter space provide complementary views of model computation. Activations represent information, while weights read, transform, and write that information. Yet existing interpretability methods largely study the two spaces separately, leaving the connection between represented information and parameter-level computation underexplored. We introduce Activation-Supported Parameter Decomposition (ASPD), which jointly decomposes activation and parameter spaces and grounds each learned weight component in the activation features it reads or writes. This grounding constrains otherwise non-unique parameter decompositions using the model's internal activations, while an internal reconstruction objective provides a local learning signal at the weight matrix being analyzed. Together, these properties enable scalable, interpretable, and causally editable parameter decomposition in pretrained large language models, demonstrated on Qwen-3-8B. The learned read--write components can also be composed into parameter-level mechanism circuits. We use ASPD to recover mechanisms underlying the classic IOI circuit and trace semantic transformations through model weights.