# Cluster-based Input Weight Initialization for Echo State Networks

arXiv: https://arxiv.org/abs/2103.04710 · HTML: https://mlsanity.com/paper/2103.04710/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 52 references that are in the map.
- First posted on arXiv 8 Mar 2021.

## Builds on (most cited first, 6 of 6)

- [Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers](https://mlsanity.com/paper/1910.00659/) (2019)
- [Reservoir computing approaches for representation and classification of multivariate time series](https://mlsanity.com/paper/1803.07870/) (2018)
- [Stochastic Configuration Networks: Fundamentals and Algorithms](https://mlsanity.com/paper/1702.03180/) (2017)
- [Deep Randomized Neural Networks](https://mlsanity.com/paper/2002.12287/) (2020)
- [A Regression Model of Recurrent Deep Neural Networks for Noise Robust Estimation of the Fundamental Frequency Contour of Speech](https://mlsanity.com/paper/1805.02958/) (2018)
- [Waveform to Single Sinusoid Regression to Estimate the F0 Contour from Noisy Speech Using Recurrent Deep Neural Networks](https://mlsanity.com/paper/1807.00752/) (2018)

## Built on by (most cited first, 4 of 4)

- [Random vector functional link network: recent developments, applications, and future directions](https://mlsanity.com/paper/2203.11316/) (2022)
- [Neural timescales from a computational perspective](https://mlsanity.com/paper/2409.02684/) (2024)
- [Neuro-Inspired Visual Pattern Recognition via Biological Reservoir Computing](https://mlsanity.com/paper/2602.05737/) (2026)
- [Masked Feature Modelling: Feature Masking for the Unsupervised Pre-training of a Graph Attention Network Block for Bottom-up Video Event Recognition](https://mlsanity.com/paper/2308.12673/) (2023)

## Abstract (arXiv)

Echo State Networks (ESNs) are a special type of recurrent neural networks (RNNs), in which the input and recurrent connections are traditionally generated randomly, and only the output weights are trained. Despite the recent success of ESNs in various tasks of audio, image and radar recognition, we postulate that a purely random initialization is not the ideal way of initializing ESNs. The aim of this work is to propose an unsupervised initialization of the input connections using the K-Means algorithm on the training data. We show that for a large variety of datasets this initialization performs equivalently or superior than a randomly initialized ESN whilst needing significantly less reservoir neurons. Furthermore, we discuss that this approach provides the opportunity to estimate a suitable size of the reservoir based on prior knowledge about the data.