# Echo state networks are universal

arXiv: https://arxiv.org/abs/1806.00797 · HTML: https://mlsanity.com/paper/1806.00797/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2018.

## Built on by (most cited first, 30 of 87)

- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics](https://mlsanity.com/paper/2103.03174/) (2021)
- [Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer](https://mlsanity.com/paper/2201.13094/) (2022)
- [Global universal approximation of functional input maps on weighted spaces](https://mlsanity.com/paper/2306.03303/) (2023)
- [Learning strange attractors with reservoir systems](https://mlsanity.com/paper/2108.05024/) (2021)
- [Neural Oscillators are Universal](https://mlsanity.com/paper/2305.08753/) (2023)
- [Universal Approximation Theorem and error bounds for quantum neural networks and quantum reservoirs](https://mlsanity.com/paper/2307.12904/) (2023)
- [A Brief Survey on the Approximation Theory for Sequence Modelling](https://mlsanity.com/paper/2302.13752/) (2023)
- [Full error analysis of the random deep splitting method for nonlinear parabolic PDEs and PIDEs](https://mlsanity.com/paper/2405.05192/) (2024)
- [Chaotic Hedging with Iterated Integrals and Neural Networks](https://mlsanity.com/paper/2209.10166/) (2022)
- [Applications of Signature Methods to Market Anomaly Detection](https://mlsanity.com/paper/2201.02441/) (2022)
- [Infinite-dimensional next-generation reservoir computing](https://mlsanity.com/paper/2412.09800/) (2024)
- [Reservoir kernels and Volterra series](https://mlsanity.com/paper/2212.14641/) (2022)
- [Recurrent Natural Policy Gradient for POMDPs](https://mlsanity.com/paper/2405.18221/) (2024)
- [Infinite-dimensional reservoir computing](https://mlsanity.com/paper/2304.00490/) (2023)
- [Stability analysis of chaotic systems in latent spaces](https://mlsanity.com/paper/2410.00480/) (2024)
- [Data-driven prediction and control of extreme events in a chaotic flow](https://mlsanity.com/paper/2204.11682/) (2022)
- [Memory of recurrent networks: Do we compute it right?](https://mlsanity.com/paper/2305.01457/) (2023)
- [Embedding Information onto a Dynamical System](https://mlsanity.com/paper/2105.10766/) (2021)
- [Universality and approximation bounds for echo state networks with random weights](https://mlsanity.com/paper/2206.05669/) (2022)
- [Learning noise-induced transitions by multi-scaling reservoir computing](https://mlsanity.com/paper/2309.05413/) (2023)
- [Positional Encoding Helps Recurrent Neural Networks Handle a Large Vocabulary](https://mlsanity.com/paper/2402.00236/) (2024)
- [How more data can hurt: Instability and regularization in next-generation reservoir computing](https://mlsanity.com/paper/2407.08641/) (2024)
- [Feedback-driven recurrent quantum neural network universality](https://mlsanity.com/paper/2506.16332/) (2025)
- [ResCP: Reservoir Conformal Prediction for Time Series Forecasting](https://mlsanity.com/paper/2510.05060/) (2025)
- [Modelling spatiotemporal turbulent dynamics with the convolutional autoencoder echo state network](https://mlsanity.com/paper/2211.11379/) (2022)
- [Stochastic Reservoir Computers](https://mlsanity.com/paper/2405.12382/) (2024)
- [Universal randomised signatures for generative time series modelling](https://mlsanity.com/paper/2406.10214/) (2024)
- [Nonlinear Autoregression with Convergent Dynamics on Novel Computational Platforms](https://mlsanity.com/paper/2108.08001/) (2021)
- [Delay Embedded Echo-State Network: A Predictor for Partially Observed Systems](https://mlsanity.com/paper/2211.05992/) (2022)

## Abstract (arXiv)

This paper shows that echo state networks are universal uniform approximants in the context of discrete-time fading memory filters with uniformly bounded inputs defined on negative infinite times. This result guarantees that any fading memory input/output system in discrete time can be realized as a simple finite-dimensional neural network-type state-space model with a static linear readout map. This approximation is valid for infinite time intervals. The proof of this statement is based on fundamental results, also presented in this work, about the topological nature of the fading memory property and about reservoir computing systems generated by continuous reservoir maps.