# Edge of stability echo state networks

arXiv: https://arxiv.org/abs/2308.02902 · HTML: https://mlsanity.com/paper/2308.02902/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 56 references that are in the map.
- First posted on arXiv 5 Aug 2023.

## Builds on (most cited first, 5 of 5)

- [Scalable Spatiotemporal Graph Neural Networks](https://mlsanity.com/paper/2209.06520/) (2022)
- [Fast and Deep Graph Neural Networks](https://mlsanity.com/paper/1911.08941/) (2019)
- [Optimizing Memory in Reservoir Computers](https://mlsanity.com/paper/2201.01605/) (2022)
- [Determination of the edge of criticality in echo state networks through Fisher information maximization](https://mlsanity.com/paper/1603.03685/) (2016)
- [Chasing the Echo State Property](https://mlsanity.com/paper/1811.10892/) (2018)

## Built on by (most cited first, 4 of 4)

- [ResCP: Reservoir Conformal Prediction for Time Series Forecasting](https://mlsanity.com/paper/2510.05060/) (2025)
- [Residual Reservoir Memory Networks](https://mlsanity.com/paper/2508.09925/) (2025)
- [Echo State Transformer: Attention Over Finite Memories](https://mlsanity.com/paper/2507.02917/) (2025)
- [Deep Residual Echo State Networks: exploring residual orthogonal connections in untrained Recurrent Neural Networks](https://mlsanity.com/paper/2508.21172/) (2025)

## Abstract (arXiv)

Echo State Networks (ESNs) are time-series processing models working under the Echo State Property (ESP) principle. The ESP is a notion of stability that imposes an asymptotic fading of the memory of the input. On the other hand, the resulting inherent architectural bias of ESNs may lead to an excessive loss of information, which in turn harms the performance in certain tasks with long short-term memory requirements. With the goal of bringing together the fading memory property and the ability to retain as much memory as possible, in this paper we introduce a new ESN architecture, called the Edge of Stability Echo State Network (ES2N). The introduced ES2N model is based on defining the reservoir layer as a convex combination of a nonlinear reservoir (as in the standard ESN), and a linear reservoir that implements an orthogonal transformation. We provide a thorough mathematical analysis of the introduced model, proving that the whole eigenspectrum of the Jacobian of the ES2N map can be contained in an annular neighbourhood of a complex circle of controllable radius, and exploit this property to demonstrate that the ES2N's forward dynamics evolves close to the edge-of-chaos regime by design. Remarkably, our experimental analysis shows that the newly introduced reservoir model is able to reach the theoretical maximum short-term memory capacity. At the same time, in comparison to standard ESN, ES2N is shown to offer an excellent trade-off between memory and nonlinearity, as well as a significant improvement of performance in autoregressive nonlinear modeling.