# A Review of Designs and Applications of Echo State Networks

arXiv: https://arxiv.org/abs/2012.02974 · HTML: https://mlsanity.com/paper/2012.02974/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2020.

## Built on by (most cited first, 11 of 11)

- [TE-ESN: Time Encoding Echo State Network for Prediction Based on Irregularly Sampled Time Series Data](https://mlsanity.com/paper/2105.00412/) (2021)
- [Learning Reservoir Dynamics with Temporal Self-Modulation](https://mlsanity.com/paper/2301.09235/) (2023)
- [Layer-wise Relevance Propagation for Echo State Networks applied to Earth System Variability](https://mlsanity.com/paper/2210.09958/) (2022)
- [Re-visiting Reservoir Computing architectures optimized by Evolutionary Algorithms](https://mlsanity.com/paper/2211.06254/) (2022)
- [Computational Complexity Evaluation of Neural Network Applications in Signal Processing](https://mlsanity.com/paper/2206.12191/) (2022)
- [Divide And Conquer: Learning Chaotic Dynamical Systems With Multistep Penalty Neural Ordinary Differential Equations](https://mlsanity.com/paper/2407.00568/) (2024)
- [Efficient Online Variational Estimation via Monte Carlo Sampling](https://mlsanity.com/paper/2602.06579/) (2026)
- [Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation](https://mlsanity.com/paper/2408.02685/) (2024)
- [Topology Structure Optimization of Reservoirs Using GLMY Homology](https://mlsanity.com/paper/2509.11612/) (2025)
- [Modeling Biological Multifunctionality with Echo State Networks](https://mlsanity.com/paper/2510.23940/) (2025)
- [Multivariate time series prediction using clustered echo state network](https://mlsanity.com/paper/2512.08963/) (2025)

## Abstract (arXiv)

Recurrent Neural Networks (RNNs) have demonstrated their outstanding ability in sequence tasks and have achieved state-of-the-art in wide range of applications, such as industrial, medical, economic and linguistic. Echo State Network (ESN) is simple type of RNNs and has emerged in the last decade as an alternative to gradient descent training based RNNs. ESN, with a strong theoretical ground, is practical, conceptually simple, easy to implement. It avoids non-converging and computationally expensive in the gradient descent methods. Since ESN was put forward in 2002, abundant existing works have promoted the progress of ESN, and the recently introduced Deep ESN model opened the way to uniting the merits of deep learning and ESNs. Besides, the combinations of ESNs with other machine learning models have also overperformed baselines in some applications. However, the apparent simplicity of ESNs can sometimes be deceptive and successfully applying ESNs needs some experience. Thus, in this paper, we categorize the ESN-based methods to basic ESNs, DeepESNs and combinations, then analyze them from the perspective of theoretical studies, network designs and specific applications. Finally, we discuss the challenges and opportunities of ESNs by summarizing the open questions and proposing possible future works.