# Training Neural Networks for Modularity aids Interpretability

arXiv: https://arxiv.org/abs/2409.15747 · HTML: https://mlsanity.com/paper/2409.15747/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Builds on (most cited first, 15 of 15)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [MoEfication: Transformer Feed-forward Layers are Mixtures of Experts](https://mlsanity.com/paper/2110.01786/) (2021)
- [Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla](https://mlsanity.com/paper/2307.09458/) (2023)
- [Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability](https://mlsanity.com/paper/2305.08746/) (2023)
- [Clusterability in Neural Networks](https://mlsanity.com/paper/2103.03386/) (2021)
- [Modular Representation of Layered Neural Networks](https://mlsanity.com/paper/1703.00168/) (2017)
- [Clustering units in neural networks: upstream vs downstream information](https://mlsanity.com/paper/2203.11815/) (2022)
- [Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks](https://mlsanity.com/paper/2310.07711/) (2023)
- [Improving the Interpretability of fMRI Decoding using Deep Neural Networks and Adversarial Robustness](https://mlsanity.com/paper/2004.11114/) (2020)
- [Challenges in Mechanistically Interpreting Model Representations](https://mlsanity.com/paper/2402.03855/) (2024)
- [Deriving pairwise transfer entropy from network structure and motifs](https://mlsanity.com/paper/1911.02931/) (2019)

## Built on by (most cited first, 1 of 1)

- [Agency Among Agents: Designing with Hypertextual Friction in the Algorithmic Web](https://mlsanity.com/paper/2507.23585/) (2025)

## Abstract (arXiv)

An approach to improve network interpretability is via clusterability, i.e., splitting a model into disjoint clusters that can be studied independently. We find pretrained models to be highly unclusterable and thus train models to be more modular using an ``enmeshment loss'' function that encourages the formation of non-interacting clusters. Using automated interpretability measures, we show that our method finds clusters that learn different, disjoint, and smaller circuits for CIFAR-10 labels. Our approach provides a promising direction for making neural networks easier to interpret.