# Model-free prediction of spatiotemporal dynamical systems with recurrent neural networks: Role of network spectral radius

arXiv: https://arxiv.org/abs/1910.04426 · HTML: https://mlsanity.com/paper/1910.04426/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2019.

## Built on by (most cited first, 30 of 34)

- [Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics](https://mlsanity.com/paper/2103.03174/) (2021)
- [Model scale versus domain knowledge in statistical forecasting of chaotic systems](https://mlsanity.com/paper/2303.08011/) (2023)
- [Anticipating synchronization with machine learning](https://mlsanity.com/paper/2103.13358/) (2021)
- [Adaptable Hamiltonian neural networks](https://mlsanity.com/paper/2102.13235/) (2021)
- [Analogue and Physical Reservoir Computing Using Water Waves](https://mlsanity.com/paper/2306.09095/) (2023)
- [Model-free tracking control of complex dynamical trajectories with machine learning](https://mlsanity.com/paper/2309.11470/) (2023)
- [Catch-22s of reservoir computing](https://mlsanity.com/paper/2210.10211/) (2022)
- [TE-ESN: Time Encoding Echo State Network for Prediction Based on Irregularly Sampled Time Series Data](https://mlsanity.com/paper/2105.00412/) (2021)
- [Embedding Theory of Reservoir Computing and Reducing Reservoir Network Using Time Delays](https://mlsanity.com/paper/2303.09042/) (2023)
- [Attractor reconstruction with reservoir computers: The effect of the reservoir's conditional Lyapunov exponents on faithful attractor reconstruction](https://mlsanity.com/paper/2401.00885/) (2023)
- [Data-driven model discovery with Kolmogorov-Arnold networks](https://mlsanity.com/paper/2409.15167/) (2024)
- [Learning noise-induced transitions by multi-scaling reservoir computing](https://mlsanity.com/paper/2309.05413/) (2023)
- [Physical Reservoir Computing Enabled by Solitary Waves and Biologically-Inspired Nonlinear Transformation of Input Data](https://mlsanity.com/paper/2402.03319/) (2024)
- [Inferring Attracting Basins of Power System with Machine Learning](https://mlsanity.com/paper/2305.14374/) (2023)
- [Machine-learning parameter tracking with partial state observation](https://mlsanity.com/paper/2311.09142/) (2023)
- [Forecasting Using Reservoir Computing: The Role of Generalized Synchronization](https://mlsanity.com/paper/2102.08930/) (2021)
- [Learning Hamiltonian dynamics by reservoir computer](https://mlsanity.com/paper/2104.14474/) (2021)
- [Scope and Sense of Explainability for AI-Systems](https://mlsanity.com/paper/2112.10551/) (2021)
- [Predicting extreme events from data using deep machine learning: when and where](https://mlsanity.com/paper/2203.17155/) (2022)
- [Digital twins of nonlinear dynamical systems](https://mlsanity.com/paper/2210.06144/) (2022)
- [Emergence of a stochastic resonance in machine learning](https://mlsanity.com/paper/2211.09955/) (2022)
- [Learning to learn ecosystems from limited data -- a meta-learning approach](https://mlsanity.com/paper/2410.07368/) (2024)
- [Reconstructing dynamics from sparse observations with no training on target system](https://mlsanity.com/paper/2410.21222/) (2024)
- [Minimal Deterministic Echo State Networks Outperform Random Reservoirs in Learning Chaotic Dynamics](https://mlsanity.com/paper/2507.06050/) (2025)
- [Digital twins of nonlinear dynamical systems: A perspective](https://mlsanity.com/paper/2309.11461/) (2023)
- [Modeling Nonlinear Oscillator Networks Using Physics-Informed Hybrid Reservoir Computing](https://mlsanity.com/paper/2411.05867/) (2024)
- [Data-driven and machine-learning based prediction of wave propagation behavior in dam-break flood](https://mlsanity.com/paper/2209.08729/) (2022)
- [Large-scale Dynamic Network Representation via Tensor Ring Decomposition](https://mlsanity.com/paper/2304.08798/) (2023)
- [A Momentum-Incorporated Non-Negative Latent Factorization of Tensors Model for Dynamic Network Representation](https://mlsanity.com/paper/2305.02782/) (2023)
- [Transforming Traditional Neural Networks into Neuromorphic Quantum-Cognitive Models: A Tutorial with Applications](https://mlsanity.com/paper/2503.07681/) (2025)

## Abstract (arXiv)

A common difficulty in applications of machine learning is the lack of any general principle for guiding the choices of key parameters of the underlying neural network. Focusing on a class of recurrent neural networks - reservoir computing systems that have recently been exploited for model-free prediction of nonlinear dynamical systems, we uncover a surprising phenomenon: the emergence of an interval in the spectral radius of the neural network in which the prediction error is minimized. In a three-dimensional representation of the error versus time and spectral radius, the interval corresponds to the bottom region of a "valley." Such a valley arises for a variety of spatiotemporal dynamical systems described by nonlinear partial differential equations, regardless of the structure and the edge-weight distribution of the underlying reservoir network. We also find that, while the particular location and size of the valley would depend on the details of the target system to be predicted, the interval tends to be larger for undirected than for directed networks. The valley phenomenon can be beneficial to the design of optimal reservoir computing, representing a small step forward in understanding these machine-learning systems.