# On the dimension of pullback attractors in recurrent neural networks

arXiv: https://arxiv.org/abs/2501.11357 · HTML: https://mlsanity.com/paper/2501.11357/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 55 references that are in the map.
- First posted on arXiv 20 Jan 2025.

## Builds on (most cited first, 11 of 11)

- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [Echo state networks are universal](https://mlsanity.com/paper/1806.00797/) (2018)
- [Can recurrent neural networks warp time?](https://mlsanity.com/paper/1804.11188/) (2018)
- [Machine learning prediction of critical transition and system collapse](https://mlsanity.com/paper/2012.01545/) (2020)
- [Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systems](https://mlsanity.com/paper/2207.00521/) (2022)
- [Fading memory echo state networks are universal](https://mlsanity.com/paper/2010.12047/) (2020)
- [Multifunctionality in a Reservoir Computer](https://mlsanity.com/paper/2008.06348/) (2020)
- [Learning strange attractors with reservoir systems](https://mlsanity.com/paper/2108.05024/) (2021)
- [Interpreting recurrent neural networks behaviour via excitable network attractors](https://mlsanity.com/paper/1807.10478/) (2018)
- [Attractor reconstruction with reservoir computers: The effect of the reservoir's conditional Lyapunov exponents on faithful attractor reconstruction](https://mlsanity.com/paper/2401.00885/) (2023)
- [Transitions in echo index and dependence on input repetitions](https://mlsanity.com/paper/2309.04728/) (2023)

## Abstract (arXiv)

Recurrent neural networks trained via the reservoir computing paradigm have demonstrated remarkable success in learning and reconstructing attractors from chaotic systems, often replicating quantities such as Lyapunov exponents and fractal dimensions. It has recently been conjectured that this is because the reservoir computer embeds the dynamics of the chaotic system in its state space before learning. This conjecture has been established for reservoir computers with linear activation functions and remains open for more general reservoir systems. In this work, we employ a non-autonomous dynamical systems approach to establish an upper bound for the box-counting dimension of the pullback attractor, a subset of the reservoir state space that is approximated during training and prediction phases. We prove that the box-counting dimension of the pullback attractor is bounded above by the box-counting dimension of the space of input sequences with respect to the product topology. In particular, for input sequences originating from an Nin-dimensional smooth dynamical system or their generic continuously differentiable observations, the box-counting dimension of the pullback attractor is bounded above by Nin. The results obtained here highlight the fact that, while a reservoir computer may possess a very high-dimensional state space, it exhibits effective low-dimensional dynamics. Our findings also partly explain why reservoir computers are successful in tasks such as attractor reconstruction and the computation of dynamic invariants like Lyapunov exponents and fractal dimensions.