# Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks

arXiv: https://arxiv.org/abs/2310.07711 · HTML: https://mlsanity.com/paper/2310.07711/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 7 references that are in the map.
- First posted on arXiv 11 Oct 2023.

## Builds on (most cited first, 2 of 2)

- [Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability](https://mlsanity.com/paper/2305.08746/) (2023)
- [Winning the lottery with neural connectivity constraints: faster learning across cognitive tasks with spatially constrained sparse RNNs](https://mlsanity.com/paper/2207.03523/) (2022)

## Built on by (most cited first, 7 of 7)

- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [KAN 2.0: Kolmogorov-Arnold Networks Meet Science](https://mlsanity.com/paper/2408.10205/) (2024)
- [Mixture of Experts Made Intrinsically Interpretable](https://mlsanity.com/paper/2503.07639/) (2025)
- [Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells](https://mlsanity.com/paper/2311.02316/) (2023)
- [Evaluating Brain-Inspired Modular Training in Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2401.03646/) (2024)
- [Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience](https://mlsanity.com/paper/2408.12664/) (2024)
- [Training Neural Networks for Modularity aids Interpretability](https://mlsanity.com/paper/2409.15747/) (2024)

## Abstract (arXiv)

Recurrent neural networks (RNNs) trained on compositional tasks can exhibit functional modularity, in which neurons can be clustered by activity similarity and participation in shared computational subtasks. Unlike brains, these RNNs do not exhibit anatomical modularity, in which functional clustering is correlated with strong recurrent coupling and spatial localization of functional clusters. Contrasting with functional modularity, which can be ephemerally dependent on the input, anatomically modular networks form a robust substrate for solving the same subtasks in the future. To examine whether it is possible to grow brain-like anatomical modularity, we apply a recent machine learning method, brain-inspired modular training (BIMT), to a network being trained to solve a set of compositional cognitive tasks. We find that functional and anatomical clustering emerge together, such that functionally similar neurons also become spatially localized and interconnected. Moreover, compared to standard L1 or no regularization settings, the model exhibits superior performance by optimally balancing task performance and network sparsity. In addition to achieving brain-like organization in RNNs, our findings also suggest that BIMT holds promise for applications in neuromorphic computing and enhancing the interpretability of neural network architectures.