mlsanityData to 14 Oct 2026

Stochastic Parameter Decomposition

arXiv 2506.20790 ↗First posted 25 Jun 2025cs.LG cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

A key step in reverse engineering neural networks is to decompose them into simpler parts that can be studied in relative isolation. Linear parameter decomposition -- a framework that has been proposed to resolve several issues with current decomposition methods -- decomposes neural network parameters into a sum of sparsely used vectors in parameter space. However, the current main method in this framework, Attribution-based Parameter Decomposition (APD), is impractical on account of its computational cost and sensitivity to hyperparameters. In this work, we introduce textitStochastic Parameter Decomposition (SPD), a method that is more scalable and robust to hyperparameters than APD, which we demonstrate by decomposing models that are slightly larger and more complex than was possible to decompose with APD. We also show that SPD avoids other issues, such as shrinkage of the learned parameters, and better identifies ground truth mechanisms in toy models. By bridging causal mediation analysis and network decomposition methods, this demonstration opens up new research possibilities in mechanistic interpretability by removing barriers to scaling linear parameter decomposition methods to larger models. We release a library for running SPD and reproducing our experiments at https://github.com/goodfire-ai/spd/tree/spd-paper.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 25 of its 41 references that are in the map. First posted on arXiv 25 Jun 2025.

Builds on

25 of 41 references.

References · 25

  1. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  2. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  4. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  5. 2018Sanity Checks for Saliency Mapscited by 1,156
  6. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  7. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  8. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  9. 2021Causal Abstractions of Neural Networkscited by 422
  10. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  11. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 209
  12. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
13 more
  1. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 156
  2. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
  3. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 108
  4. 2020Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Maskscited by 98
  5. 2025Sparse Autoencoders Do Not Find Canonical Units of Analysiscited by 88
  6. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
  7. 2021Low-Complexity Probing via Finding Subnetworkscited by 64
  8. 2024Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficientcited by 40
  9. 2024Mathematical Models of Computation in Superpositioncited by 35
  10. 2025Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decompositioncited by 27
  11. 2024Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networkscited by 23
  12. 2025Identifying Sparsely Active Circuits Through Local Loss Landscape Decompositioncited by 5
  13. 2023Uncovering Model Processing Strategies with Non-Negative Per-Example Fisher Factorizationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 10.

16 papers.

Sort:

By area:

  1. 2025Weight-sparse transformers have interpretable circuitscited by 44
  2. 2025Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learningcited by 8
  3. 2025Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMscited by 7
  4. 2026Bi-Orthogonal Factor Decomposition for Vision Transformerscited by 5
  5. 2025From Memorization to Reasoning in the Spectrum of Loss Curvaturecited by 2
  6. 2025Hedonic Neurons: A Mechanistic Mapping of Latent Coalitions in Transformer MLPscited by 1
  7. 2025Decomposition of Small Transformer Modelscited by 1
  8. 2026From Mechanistic to Compositional Interpretabilitycited by 1
  9. 2026Targeted Recovery of Weight-Space Mechanisms From Neural Networkscited by 1
  10. 2026Sparse Weight Decomposition for Efficient Circuit Extractioncited by 1
  11. 2026Matryoshka attribution: Learning to attribute language model outputs to representations and weightscited by 1
  12. 2026From Weights to Concepts: Data-Free Interpretability of CLIP via Singular Vector Decompositioncited by 0
4 more
  1. 2026AI Engram: In Search of Memory Traces in Artificial Intelligencecited by 0
  2. 2026Individual Parameters in Weight-Sparse Transformers Appear Interpretablecited by 0
  3. 2026Compressed Computation under L4 Loss is likely Computation in Superpositioncited by 0
  4. 2026Weights Read and Write Features: Scalable Parameter Decomposition Grounded in Activation Spacecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Interpreting Language Model Parameters
  2. 2025Descend or Rewind? Stochastic Gradient Descent Unlearning
  3. 2025Stochastic Forward-Backward Deconvolution: Training Diffusion Models with Finite Noisy Datasets
  4. 2025Combining Causal Models for More Accurate Abstractions of Neural Networks
  5. 2024Probabilistic Decomposed Linear Dynamical Systems for Robust Discovery of Latent Neural Dynamics
  6. 2024Over-parameterized Student Model via Tensor Decomposition Boosted Knowledge Distillation
  7. 2025Overparametrization bends the landscape: BBP transitions at initialization in simple Neural Networks
  8. 2024MUSO: Achieving Exact Machine Unlearning in Over-Parameterized Regimes