# Catch-22s of reservoir computing

arXiv: https://arxiv.org/abs/2210.10211 · HTML: https://mlsanity.com/paper/2210.10211/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 78 references that are in the map.
- First posted on arXiv 18 Oct 2022.

## Builds on (most cited first, 30 of 31)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Modern Koopman Theory for Dynamical Systems](https://mlsanity.com/paper/2102.12086/) (2021)
- [Next Generation Reservoir Computing](https://mlsanity.com/paper/2106.07688/) (2021)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation](https://mlsanity.com/paper/2105.14368/) (2021)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [A Bayesian machine scientist to aid in the solution of challenging scientific problems](https://mlsanity.com/paper/2004.12157/) (2020)
- [Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model](https://mlsanity.com/paper/1803.04779/) (2018)
- [Machine learning prediction of critical transition and system collapse](https://mlsanity.com/paper/2012.01545/) (2020)
- [Reservoir Computing Universality With Stochastic Inputs](https://mlsanity.com/paper/1807.02621/) (2018)
- [Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers](https://mlsanity.com/paper/1910.00659/) (2019)
- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [Model-free prediction of spatiotemporal dynamical systems with recurrent neural networks: Role of network spectral radius](https://mlsanity.com/paper/1910.04426/) (2019)
- [Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systems](https://mlsanity.com/paper/2207.00521/) (2022)
- [Combining Machine Learning with Knowledge-Based Modeling for Scalable Forecasting and Subgrid-Scale Closure of Large, Complex, Spatiotemporal Systems](https://mlsanity.com/paper/2002.05514/) (2020)
- [Anticipating synchronization with machine learning](https://mlsanity.com/paper/2103.13358/) (2021)
- [Echo State Networks trained by Tikhonov least squares are L2(μ) approximators of ergodic dynamical systems](https://mlsanity.com/paper/2005.06967/) (2020)
- [Multifunctionality in a Reservoir Computer](https://mlsanity.com/paper/2008.06348/) (2020)
- [Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations](https://mlsanity.com/paper/2108.03561/) (2021)
- [Model-free inference of unseen attractors: Reconstructing phase space features from a single noisy trajectory using reservoir computing](https://mlsanity.com/paper/2108.04074/) (2021)
- [Do Reservoir Computers Work Best at the Edge of Chaos?](https://mlsanity.com/paper/2012.01409/) (2020)
- [Learning strange attractors with reservoir systems](https://mlsanity.com/paper/2108.05024/) (2021)
- [Parallel Machine Learning for Forecasting the Dynamics of Complex Networks](https://mlsanity.com/paper/2108.12129/) (2021)
- [Using a reservoir computer to learn chaotic attractors, with applications to chaos synchronisation and cryptography](https://mlsanity.com/paper/1802.02844/) (2018)
- [Rapid Time Series Prediction with a Hardware-Based Reservoir Computer](https://mlsanity.com/paper/1807.07627/) (2018)
- [Optimizing Memory in Reservoir Computers](https://mlsanity.com/paper/2201.01605/) (2022)
- [Learning unseen coexisting attractors](https://mlsanity.com/paper/2207.14133/) (2022)
- [Machine Learning Link Inference of Noisy Delay-coupled Networks with Opto-Electronic Experimental Tests](https://mlsanity.com/paper/2010.15289/) (2020)

## Built on by (most cited first, 11 of 11)

- [Zero-shot forecasting of chaotic systems](https://mlsanity.com/paper/2409.15771/) (2024)
- [Model scale versus domain knowledge in statistical forecasting of chaotic systems](https://mlsanity.com/paper/2303.08011/) (2023)
- [How more data can hurt: Instability and regularization in next-generation reservoir computing](https://mlsanity.com/paper/2407.08641/) (2024)
- [On the emergence of numerical instabilities in Next Generation Reservoir Computing](https://mlsanity.com/paper/2505.00846/) (2025)
- [Learning to learn ecosystems from limited data -- a meta-learning approach](https://mlsanity.com/paper/2410.07368/) (2024)
- [Tailored Forecasting from Short Time Series via Meta-learning](https://mlsanity.com/paper/2501.16325/) (2025)
- [Learning Beyond Experience: Generalizing to Unseen State Space with Reservoir Computing](https://mlsanity.com/paper/2506.05292/) (2025)
- [Modeling Nonlinear Oscillator Networks Using Physics-Informed Hybrid Reservoir Computing](https://mlsanity.com/paper/2411.05867/) (2024)
- [Tailored minimal reservoir computing: on the bidirectional connection between nonlinearities in the reservoir and in data](https://mlsanity.com/paper/2504.17503/) (2025)
- [Flow map learning in nonlinear vector autoregressive models: influence of the feature-library structure on the training error](https://mlsanity.com/paper/2605.31438/) (2026)
- [Causal Local States: Scalable Simultaneous Causal Network Inference and Forecasting for Dynamical Systems](https://mlsanity.com/paper/2608.17452/) (2026)

## Abstract (arXiv)

Reservoir Computing (RC) is a simple and efficient model-free framework for forecasting the behavior of nonlinear dynamical systems from data. Here, we show that there exist commonly-studied systems for which leading RC frameworks struggle to learn the dynamics unless key information about the underlying system is already known. We focus on the important problem of basin prediction -- determining which attractor a system will converge to from its initial conditions. First, we show that the predictions of standard RC models (echo state networks) depend critically on warm-up time, requiring a warm-up trajectory containing almost the entire transient in order to identify the correct attractor. Accordingly, we turn to Next-Generation Reservoir Computing (NGRC), an attractive variant of RC that requires negligible warm-up time. By incorporating the exact nonlinearities in the original equations, we show that NGRC can accurately reconstruct intricate and high-dimensional basins of attraction, even with sparse training data (e.g., a single transient trajectory). Yet, a tiny uncertainty in the exact nonlinearity can render prediction accuracy no better than chance. Our results highlight the challenges faced by data-driven methods in learning the dynamics of multistable systems and suggest potential avenues to make these approaches more robust.