# Neuronal correlations shape the scaling behavior of memory capacity and nonlinear computational capability of reservoir recurrent neural networks

arXiv: https://arxiv.org/abs/2504.19657 · HTML: https://mlsanity.com/paper/2504.19657/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 65 references that are in the map.
- First posted on arXiv 28 Apr 2025.

## Builds on (most cited first, 15 of 15)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks](https://mlsanity.com/paper/2205.09653/) (2022)
- [Generative Artificial Intelligence: A Systematic Review and Applications](https://mlsanity.com/paper/2405.11029/) (2024)
- [Theory of gating in recurrent neural networks](https://mlsanity.com/paper/2007.14823/) (2020)
- [Do Reservoir Computers Work Best at the Edge of Chaos?](https://mlsanity.com/paper/2012.01409/) (2020)
- [Optimal short-term memory before the edge of chaos in driven random recurrent networks](https://mlsanity.com/paper/1912.11213/) (2019)
- [Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings](https://mlsanity.com/paper/2305.08459/) (2023)
- [Correlations between synapses in pairs of neurons slow down dynamics in randomly connected neural networks](https://mlsanity.com/paper/1707.08337/) (2017)
- [Connectivity structure and dynamics of nonlinear recurrent neural networks](https://mlsanity.com/paper/2409.01969/) (2024)
- [Distributions of covariances as a window into the operational regime of neuronal networks](https://mlsanity.com/paper/1605.04153/) (2016)
- [Input correlations impede suppression of chaos and learning in balanced rate networks](https://mlsanity.com/paper/2201.09916/) (2022)
- [Suppression of chaos in a partially driven recurrent neural network](https://mlsanity.com/paper/2306.00900/) (2023)
- [Dimension of activity in random neural networks](https://mlsanity.com/paper/2207.12373/) (2022)

## Abstract (arXiv)

Reservoir computing is a powerful framework for real-time information processing, characterized by its high computational ability and quick learning, with applications ranging from machine learning to biological systems. In this paper, we investigate how the computational ability of reservoir recurrent neural networks (RNNs) scales with an increasing number of readout neurons. First, we demonstrate that the memory capacity of a reservoir RNN scales sublinearly with the number of readout neurons. To elucidate this observation, we develop a theoretical framework for analytically deriving memory capacity that incorporates the effect of neuronal correlations, which have been ignored in prior theoretical work for analytical simplicity. Our theory successfully relates the sublinear scaling of memory capacity to the strength of neuronal correlations. Furthermore, we show this principle holds across diverse types of RNNs, even those beyond the direct applicability of our theory. Next, we numerically investigate the scaling behavior of nonlinear computational ability, which, alongside memory capacity, is crucial for overall computational performance. Our numerical simulations reveal that as memory capacity growth becomes sublinear, increasing the number of readout neurons successively enables nonlinear processing at progressively higher polynomial orders. Our theoretical framework suggests that neuronal correlations govern not only memory capacity but also the sequential growth of nonlinear computational capabilities. Our findings establish a foundation for designing scalable and cost-effective reservoir computing, providing novel insights into the interplay among neuronal correlations, linear memory, and nonlinear processing.