# Efficient Optimization of Echo State Networks for Time Series Datasets

arXiv: https://arxiv.org/abs/1903.05071 · HTML: https://mlsanity.com/paper/1903.05071/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2019.

## Built on by (most cited first, 10 of 10)

- [Unsupervised Reservoir Computing for Solving Ordinary Differential Equations](https://mlsanity.com/paper/2108.11417/) (2021)
- [RcTorch: a PyTorch Reservoir Computing Package with Automated Hyper-Parameter Optimization](https://mlsanity.com/paper/2207.05870/) (2022)
- [Imposing Connectome-Derived Topology on an Echo State Network](https://mlsanity.com/paper/2201.09359/) (2022)
- [Using Connectome Features to Constrain Echo State Networks](https://mlsanity.com/paper/2206.02094/) (2022)
- [Evolutionary Echo State Network: evolving reservoirs in the Fourier space](https://mlsanity.com/paper/2206.04951/) (2022)
- [Re-visiting Reservoir Computing architectures optimized by Evolutionary Algorithms](https://mlsanity.com/paper/2211.06254/) (2022)
- [Echo State Networks for Time Series Forecasting: Hyperparameter Sweep and Benchmarking](https://mlsanity.com/paper/2602.03912/) (2026)
- [A Random Matrix Perspective of Echo State Networks: From Precise Bias--Variance Characterization to Optimal Regularization](https://mlsanity.com/paper/2509.22011/) (2025)
- [Free-Probability Kernels for Zero-Rollout Hyperparameter Selection in Reservoir Computing](https://mlsanity.com/paper/2608.20998/) (2026)
- [Inference of Unknown Dynamical Components Using Next Generation Reservoir Computing: From Chaotic Systems to Climate Data](https://mlsanity.com/paper/2609.24754/) (2026)

## Abstract (arXiv)

Echo State Networks (ESNs) are recurrent neural networks that only train their output layer, thereby precluding the need to backpropagate gradients through time, which leads to significant computational gains. Nevertheless, a common issue in ESNs is determining its hyperparameters, which are crucial in instantiating a well performing reservoir, but are often set manually or using heuristics. In this work we optimize the ESN hyperparameters using Bayesian optimization which, given a limited budget of function evaluations, outperforms a grid search strategy. In the context of large volumes of time series data, such as light curves in the field of astronomy, we can further reduce the optimization cost of ESNs. In particular, we wish to avoid tuning hyperparameters per individual time series as this is costly; instead, we want to find ESNs with hyperparameters that perform well not just on individual time series but rather on groups of similar time series without sacrificing predictive performance significantly. This naturally leads to a notion of clusters, where each cluster is represented by an ESN tuned to model a group of time series of similar temporal behavior. We demonstrate this approach both on synthetic datasets and real world light curves from the MACHO survey. We show that our approach results in a significant reduction in the number of ESN models required to model a whole dataset, while retaining predictive performance for the series in each cluster.