# Fading memory echo state networks are universal

arXiv: https://arxiv.org/abs/2010.12047 · HTML: https://mlsanity.com/paper/2010.12047/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2020.

## Built on by (most cited first, 30 of 32)

- [Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics](https://mlsanity.com/paper/2103.03174/) (2021)
- [Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer](https://mlsanity.com/paper/2201.13094/) (2022)
- [Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis](https://mlsanity.com/paper/2210.13300/) (2022)
- [Learning strange attractors with reservoir systems](https://mlsanity.com/paper/2108.05024/) (2021)
- [Universal Approximation Theorem and error bounds for quantum neural networks and quantum reservoirs](https://mlsanity.com/paper/2307.12904/) (2023)
- [A Brief Survey on the Approximation Theory for Sequence Modelling](https://mlsanity.com/paper/2302.13752/) (2023)
- [Inverse Approximation Theory for Nonlinear Recurrent Neural Networks](https://mlsanity.com/paper/2305.19190/) (2023)
- [Infinite-dimensional next-generation reservoir computing](https://mlsanity.com/paper/2412.09800/) (2024)
- [Reservoir kernels and Volterra series](https://mlsanity.com/paper/2212.14641/) (2022)
- [Infinite-dimensional reservoir computing](https://mlsanity.com/paper/2304.00490/) (2023)
- [Memory of recurrent networks: Do we compute it right?](https://mlsanity.com/paper/2305.01457/) (2023)
- [Universality and approximation bounds for echo state networks with random weights](https://mlsanity.com/paper/2206.05669/) (2022)
- [Feedback-driven recurrent quantum neural network universality](https://mlsanity.com/paper/2506.16332/) (2025)
- [State-space systems as dynamic generative models](https://mlsanity.com/paper/2404.08717/) (2024)
- [Role of scrambling and noise in temporal information processing with quantum systems](https://mlsanity.com/paper/2505.10080/) (2025)
- [Stochastic dynamics learning with state-space systems](https://mlsanity.com/paper/2508.07876/) (2025)
- [Reservoir Computing with Error Correction: Long-term Behaviors of Stochastic Dynamical Systems](https://mlsanity.com/paper/2305.00669/) (2023)
- [Approximation Rates and VC-Dimension Bounds for (P)ReLU MLP Mixture of Experts](https://mlsanity.com/paper/2402.03460/) (2024)
- [On the choice of the non-trainable internal weights in random feature maps](https://mlsanity.com/paper/2408.03626/) (2024)
- [Fading memory and the convolution theorem](https://mlsanity.com/paper/2408.07386/) (2024)
- [Universal Time-Uniform Trajectory Approximation for Random Dynamical Systems with Recurrent Neural Networks](https://mlsanity.com/paper/2211.08018/) (2022)
- [Refined Kolmogorov Complexity of Analog, Evolving and Stochastic Recurrent Neural Networks](https://mlsanity.com/paper/2309.17032/) (2023)
- [Universality of reservoir systems with recurrent neural networks](https://mlsanity.com/paper/2403.01900/) (2024)
- [On Dominant Manifolds in Reservoir Computing Networks](https://mlsanity.com/paper/2604.05967/) (2026)
- [Change Point Detection with Conceptors](https://mlsanity.com/paper/2308.06213/) (2023)
- [On the dimension of pullback attractors in recurrent neural networks](https://mlsanity.com/paper/2501.11357/) (2025)
- [A tensor network approach for chaotic time series prediction](https://mlsanity.com/paper/2505.17740/) (2025)
- [Fractal and Chaotic Activation Functions in Echo State Networks: Preprocessing Topology Governs the Echo State Property](https://mlsanity.com/paper/2512.14675/) (2025)
- [Data-Specific Hyper-Parameter Design: A Paradigm Shift in Reservoir Computing](https://mlsanity.com/paper/2605.25221/) (2026)
- [Kolmogorov-Arnold Reservoir Computing](https://mlsanity.com/paper/2606.19984/) (2026)

## Abstract (arXiv)

Echo state networks (ESNs) have been recently proved to be universal approximants for input/output systems with respect to various L p-type criteria. When 1≤ p< ∞, only p-integrability hypotheses need to be imposed, while in the case p=∞ a uniform boundedness hypotheses on the inputs is required. This note shows that, in the last case, a universal family of ESNs can be constructed that contains exclusively elements that have the echo state and the fading memory properties. This conclusion could not be drawn with the results and methods available so far in the literature.