# On the Statistical Challenges of Echo State Networks and Some Potential Remedies

arXiv: https://arxiv.org/abs/1802.07369 · HTML: https://mlsanity.com/paper/1802.07369/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2018.

## Built on by (most cited first, 7 of 7)

- [On the Post-hoc Explainability of Deep Echo State Networks for Time Series Forecasting, Image and Video Classification](https://mlsanity.com/paper/2102.08634/) (2021)
- [Imposing Connectome-Derived Topology on an Echo State Network](https://mlsanity.com/paper/2201.09359/) (2022)
- [Using Connectome Features to Constrain Echo State Networks](https://mlsanity.com/paper/2206.02094/) (2022)
- [Computational Complexity Evaluation of Neural Network Applications in Signal Processing](https://mlsanity.com/paper/2206.12191/) (2022)
- [Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation](https://mlsanity.com/paper/2408.02685/) (2024)
- [Empirical Investigation into Configuring Echo State Networks for Representative Benchmark Problem Domains](https://mlsanity.com/paper/2508.10887/) (2025)
- [A Random Matrix Perspective of Echo State Networks: From Precise Bias--Variance Characterization to Optimal Regularization](https://mlsanity.com/paper/2509.22011/) (2025)

## Abstract (arXiv)

Echo state networks are powerful recurrent neural networks. However, they are often unstable and shaky, making the process of finding an good ESN for a specific dataset quite hard. Obtaining a superb accuracy by using the Echo State Network is a challenging task. We create, develop and implement a family of predictably optimal robust and stable ensemble of Echo State Networks via regularizing the training and perturbing the input. Furthermore, several distributions of weights have been tried based on the shape to see if the shape of the distribution has the impact for reducing the error. We found ESN can track in short term for most dataset, but it collapses in the long run. Short-term tracking with large size reservoir enables ESN to perform strikingly with superior prediction. Based on this scenario, we go a further step to aggregate many of ESNs into an ensemble to lower the variance and stabilize the system by stochastic replications and bootstrapping of input data.