# Emergence of Functionally Differentiated Structures via Mutual Information Minimization in Recurrent Neural Networks

arXiv: https://arxiv.org/abs/2507.12858 · HTML: https://mlsanity.com/paper/2507.12858/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2025.

## Built on by (most cited first, 2 of 2)

- [One Model, Two Roles: Emergent Specialization in a Shared Recurrent Transformer](https://mlsanity.com/paper/2605.17811/) (2026)
- [Compression Through Time: An Interplay of Separability, Stationarity and Capacity](https://mlsanity.com/paper/2610.16660/) (2026)

## Abstract (arXiv)

Functional differentiation in the brain emerges as distinct regions specialize and is key to understanding brain function as a complex system. Previous research has modeled this process using artificial neural networks with specific constraints. Here, we propose a novel approach that induces functional differentiation in recurrent neural networks by minimizing mutual information between neural subgroups via mutual information neural estimation. We apply our method to a 2-bit working memory task and a chaotic signal separation task involving Lorenz and R"ossler time series. Analysis of network performance, correlation patterns, and weight matrices reveals that mutual information minimization yields high task performance alongside clear functional modularity and moderate structural modularity. Importantly, our results show that functional differentiation, which is measured through correlation structures, emerges earlier than structural modularity defined by synaptic weights. This suggests that functional specialization precedes and probably drives structural reorganization within developing neural networks. Our findings provide new insights into how information-theoretic principles may govern the emergence of specialized functions and modular structures during artificial and biological brain development.