# Using Degeneracy in the Loss Landscape for Mechanistic Interpretability

arXiv: https://arxiv.org/abs/2405.10927 · HTML: https://mlsanity.com/paper/2405.10927/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 17 May 2024.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 20 of 20)

- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Essentially No Barriers in Neural Network Energy Landscape](https://mlsanity.com/paper/1803.00885/) (2018)
- [Sensitivity and Generalization in Neural Networks: an Empirical Study](https://mlsanity.com/paper/1802.08760/) (2018)
- [A Kronecker-factored approximate Fisher matrix for convolution layers](https://mlsanity.com/paper/1602.01407/) (2016)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks](https://mlsanity.com/paper/1602.03616/) (2016)
- [Fisher-Rao Metric, Geometry, and Complexity of Neural Networks](https://mlsanity.com/paper/1711.01530/) (2017)
- [EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis](https://mlsanity.com/paper/1905.05934/) (2019)
- [The Local Learning Coefficient: A Singularity-Aware Complexity Measure](https://mlsanity.com/paper/2308.12108/) (2023)
- [Is SGD a Bayesian sampler? Well, almost](https://mlsanity.com/paper/2006.15191/) (2020)
- [Deep Learning is Singular, and That's Good](https://mlsanity.com/paper/2010.11560/) (2020)
- [Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability](https://mlsanity.com/paper/2305.08746/) (2023)
- [Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition](https://mlsanity.com/paper/2310.06301/) (2023)
- [Formalizing the presumption of independence](https://mlsanity.com/paper/2211.06738/) (2022)

## Built on by (most cited first, 13 of 13)

- [Identifying Sparsely Active Circuits Through Local Loss Landscape Decomposition](https://mlsanity.com/paper/2504.00194/) (2025)
- [Cluster-norm for Unsupervised Probing of Knowledge](https://mlsanity.com/paper/2407.18712/) (2024)
- [Feature Identification via the Empirical NTK](https://mlsanity.com/paper/2510.00468/) (2025)
- [Associative memory inspires improvements for in-context learning using a novel attention residual stream architecture](https://mlsanity.com/paper/2412.15113/) (2024)
- [Weight-based Decomposition: A Case for Bilinear MLPs](https://mlsanity.com/paper/2406.03947/) (2024)
- [Review and Prospect of Algebraic Research in Equivalent Framework between Statistical Mechanics and Machine Learning Theory](https://mlsanity.com/paper/2406.10234/) (2024)
- [From Memorization to Reasoning in the Spectrum of Loss Curvature](https://mlsanity.com/paper/2510.24256/) (2025)
- [Transformers converge to invariant algorithmic cores](https://mlsanity.com/paper/2602.22600/) (2026)
- [Dead Directions: Geometric Singular Learning](https://mlsanity.com/paper/2606.05957/) (2026)
- [NGD converges to less degenerate solutions than SGD](https://mlsanity.com/paper/2409.04913/) (2024)
- [Deep Model Merging: The Sister of Neural Network Interpretability -- A Survey](https://mlsanity.com/paper/2410.12927/) (2024)
- [Measuring Dead Directions: Decomposing and Classifying Singular Structure off Canonical Alignment](https://mlsanity.com/paper/2607.00603/) (2026)
- [Functional Degeneracy in Neural Networks: Measurement and Pruning](https://mlsanity.com/paper/2608.30741/) (2026)

## Abstract (arXiv)

Mechanistic Interpretability aims to reverse engineer the algorithms implemented by neural networks by studying their weights and activations. An obstacle to reverse engineering neural networks is that many of the parameters inside a network are not involved in the computation being implemented by the network. These degenerate parameters may obfuscate internal structure. Singular learning theory teaches us that neural network parameterizations are biased towards being more degenerate, and parameterizations with more degeneracy are likely to generalize further. We identify 3 ways that network parameters can be degenerate: linear dependence between activations in a layer; linear dependence between gradients passed back to a layer; ReLUs which fire on the same subset of datapoints. We also present a heuristic argument that modular networks are likely to be more degenerate, and we develop a metric for identifying modules in a network that is based on this argument. We propose that if we can represent a neural network in a way that is invariant to reparameterizations that exploit the degeneracies, then this representation is likely to be more interpretable, and we provide some evidence that such a representation is likely to have sparser interactions. We introduce the Interaction Basis, a tractable technique to obtain a representation that is invariant to degeneracies from linear dependence of activations or Jacobians.