# On the emergence of numerical instabilities in Next Generation Reservoir Computing

arXiv: https://arxiv.org/abs/2505.00846 · HTML: https://mlsanity.com/paper/2505.00846/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 74 references that are in the map.
- First posted on arXiv 1 May 2025.

## Builds on (most cited first, 14 of 14)

- [Physics-informed learning of governing equations from scarce data](https://mlsanity.com/paper/2005.03448/) (2020)
- [Next Generation Reservoir Computing](https://mlsanity.com/paper/2106.07688/) (2021)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [Echo State Networks trained by Tikhonov least squares are L2(μ) approximators of ergodic dynamical systems](https://mlsanity.com/paper/2005.06967/) (2020)
- [Learning Spatiotemporal Chaos Using Next-Generation Reservoir Computing](https://mlsanity.com/paper/2203.13294/) (2022)
- [Learning strange attractors with reservoir systems](https://mlsanity.com/paper/2108.05024/) (2021)
- [Learning unseen coexisting attractors](https://mlsanity.com/paper/2207.14133/) (2022)
- [Catch-22s of reservoir computing](https://mlsanity.com/paper/2210.10211/) (2022)
- [Controlling Chaos Using Edge Computing Hardware](https://mlsanity.com/paper/2406.12876/) (2024)
- [Infinite-dimensional next-generation reservoir computing](https://mlsanity.com/paper/2412.09800/) (2024)
- [Symmetry-Aware Reservoir Computing](https://mlsanity.com/paper/2102.00310/) (2021)
- [Controlling Chaotic Maps using Next-Generation Reservoir Computing](https://mlsanity.com/paper/2307.03813/) (2023)
- [Hybridizing Traditional and Next-Generation Reservoir Computing to Accurately and Efficiently Forecast Dynamical Systems](https://mlsanity.com/paper/2403.18953/) (2024)
- [How more data can hurt: Instability and regularization in next-generation reservoir computing](https://mlsanity.com/paper/2407.08641/) (2024)

## Built on by (most cited first, 3 of 3)

- [Machine-Precision Prediction of Low-Dimensional Chaotic Systems from Noise-Free Data](https://mlsanity.com/paper/2507.09652/) (2025)
- [Next-Generation Reservoir Computing for Dynamical Inference](https://mlsanity.com/paper/2509.11338/) (2025)
- [Flow map learning in nonlinear vector autoregressive models: influence of the feature-library structure on the training error](https://mlsanity.com/paper/2605.31438/) (2026)

## Abstract (arXiv)

Next Generation Reservoir Computing (NGRC) is a low-cost machine learning method for forecasting chaotic time series from data. Computational efficiency is crucial for scalable reservoir computing, requiring better strategies to reduce training cost. In this work, we uncover a connection between the numerical conditioning of the NGRC feature matrix -- formed by polynomial evaluations on time-delay coordinates -- and the long-term NGRC dynamics. We show that NGRC can be trained without regularization, reducing computational time. Our contributions are twofold. First, merging tools from numerical linear algebra and ergodic theory of dynamical systems, we systematically study how the feature matrix conditioning varies across hyperparameters. We demonstrate that the NGRC feature matrix tends to be ill-conditioned for short time lags, high-degree polynomials, and short length of training data. Second, we evaluate the impact of different numerical algorithms (Cholesky, singular value decomposition (SVD), and lower-upper (LU) decomposition) for solving the regularized least-squares problem. Our results reveal that SVD-based training achieves accurate forecasts without regularization, being preferable when compared against the other algorithms.