mlsanityData to 14 Oct 2026

Identifying Sparsely Active Circuits Through Local Loss Landscape Decomposition

arXiv 2504.00194 ↗First posted 31 Mar 2025cs.LG cs.AIcited by 5

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

Much of mechanistic interpretability has focused on understanding the activation spaces of large neural networks. However, activation space-based approaches reveal little about the underlying circuitry used to compute features. To better understand the circuits employed by models, we introduce a new decomposition method called Local Loss Landscape Decomposition (L3D). L3D identifies a set of low-rank subnetworks: directions in parameter space of which a subset can reconstruct the gradient of the loss between any sample's output and a reference output vector. We design a series of progressively more challenging toy models with well-defined subnetworks and show that L3D can nearly perfectly recover the associated subnetworks. Additionally, we investigate the extent to which perturbing the model in the direction of a given subnetwork affects only the relevant subset of samples. Finally, we apply L3D to a real-world transformer model and a convolutional neural network, demonstrating its potential to identify interpretable and relevant circuits in parameter space.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 22 of its 34 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

22 of 34 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition24 Jan
  2. 2024Using Degeneracy in the Loss Landscape for Mechanistic Interpretability17 May
  3. 2024Mechanistic Interpretability for AI Safety -- A Review22 Apr
  4. 2023Attribution Patching Outperforms Automated Circuit Discovery16 Oct

Other references · 18

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  4. 2023Steering Language Models With Activation Engineeringcited by 922
  5. 2017Empirical Analysis of the Hessian of Over-Parametrized Neural Networkscited by 290
  6. 2014Identifying and attacking the saddle point problem in high-dimensional non-convex optimizationcited by 289
  7. 2014The Loss Surfaces of Multilayer Networkscited by 280
  8. 2016Deep Learning without Poor Local Minimacited by 234
  9. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 225
  10. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  11. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
  12. 2023The Local Learning Coefficient: A Singularity-Aware Complexity Measurecited by 51
6 more
  1. 2020Deep Learning is Singular, and That's Goodcited by 49
  2. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 45
  3. 2024Talking Heads: Understanding Inter-layer Communication in Transformer Language Modelscited by 41
  4. 2024Loss Landscape Degeneracy and Stagewise Development in Transformerscited by 40
  5. 2017Exponentially vanishing sub-optimal local minima in multilayer neural networkscited by 19
  6. 2023Uncovering Model Processing Strategies with Non-Negative Per-Example Fisher Factorizationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Stochastic Parameter Decompositioncited by 16
  2. 2026Bi-Orthogonal Factor Decomposition for Vision Transformerscited by 5
  3. 2025Decomposition of Small Transformer Modelscited by 1
  4. 2026Individual Parameters in Weight-Sparse Transformers Appear Interpretablecited by 0
  5. 2026Weights Read and Write Features: Scalable Parameter Decomposition Grounded in Activation Spacecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Automatically Identifying Local and Global Circuits with Linear Computation Graphs
  2. 2024Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition
  3. 2026Sparse Weight Decomposition for Efficient Circuit Extraction
  4. 2025Circuit Insights: Towards Interpretability Beyond Activations
  5. 2024Transcoders Find Interpretable LLM Feature Circuits
  6. 2025Discovering Transformer Circuits via a Hybrid Attribution and Pruning Framework
  7. 2023Towards Automated Circuit Discovery for Mechanistic Interpretability
  8. 2026Hierarchical Sparse Circuit Extraction from Billion-Parameter Language Models through Scalable Attribution Graph Decomposition