# Differentiable reservoir computing

arXiv: https://arxiv.org/abs/1902.06094 · HTML: https://mlsanity.com/paper/1902.06094/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2019.

## Built on by (most cited first, 20 of 20)

- [Universal Approximation Under Constraints is Possible with Transformers](https://mlsanity.com/paper/2110.03303/) (2021)
- [Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer](https://mlsanity.com/paper/2201.13094/) (2022)
- [A Brief Survey on the Approximation Theory for Sequence Modelling](https://mlsanity.com/paper/2302.13752/) (2023)
- [Reservoir kernels and Volterra series](https://mlsanity.com/paper/2212.14641/) (2022)
- [Infinite-dimensional reservoir computing](https://mlsanity.com/paper/2304.00490/) (2023)
- [Universal approximation results for neural networks with non-polynomial activation function over non-compact domains](https://mlsanity.com/paper/2410.14759/) (2024)
- [Do ReLU Networks Have An Edge When Approximating Compactly-Supported Functions?](https://mlsanity.com/paper/2204.11231/) (2022)
- [Embedding Information onto a Dynamical System](https://mlsanity.com/paper/2105.10766/) (2021)
- [Universality and approximation bounds for echo state networks with random weights](https://mlsanity.com/paper/2206.05669/) (2022)
- [How more data can hurt: Instability and regularization in next-generation reservoir computing](https://mlsanity.com/paper/2407.08641/) (2024)
- [ResCP: Reservoir Conformal Prediction for Time Series Forecasting](https://mlsanity.com/paper/2510.05060/) (2025)
- [State-space systems as dynamic generative models](https://mlsanity.com/paper/2404.08717/) (2024)
- [Low-dimensional approximations of the conditional law of Volterra processes: a non-positive curvature approach](https://mlsanity.com/paper/2405.20094/) (2024)
- [RandNet-Parareal: a time-parallel PDE solver using Random Neural Networks](https://mlsanity.com/paper/2411.06225/) (2024)
- [Stochastic dynamics learning with state-space systems](https://mlsanity.com/paper/2508.07876/) (2025)
- [Fading memory and the convolution theorem](https://mlsanity.com/paper/2408.07386/) (2024)
- [Echoes of the Past: A Unified Perspective on Fading memory and Echo States](https://mlsanity.com/paper/2508.19145/) (2025)
- [Interpretable Design of Reservoir Computing Networks using Realization Theory](https://mlsanity.com/paper/2112.06891/) (2021)
- [Using Echo State Networks to Approximate Value Functions for Control](https://mlsanity.com/paper/2102.06258/) (2021)
- [A tensor network approach for chaotic time series prediction](https://mlsanity.com/paper/2505.17740/) (2025)

## Abstract (arXiv)

Much effort has been devoted in the last two decades to characterize the situations in which a reservoir computing system exhibits the so-called echo state (ESP) and fading memory (FMP) properties. These important features amount, in mathematical terms, to the existence and continuity of global reservoir system solutions. That research is complemented in this paper with the characterization of the differentiability of reservoir filters for very general classes of discrete-time deterministic inputs. This constitutes a novel strong contribution to the long line of research on the ESP and the FMP and, in particular, links to existing research on the input-dependence of the ESP. Differentiability has been shown in the literature to be a key feature in the learning of attractors of chaotic dynamical systems. A Volterra-type series representation for reservoir filters with semi-infinite discrete-time inputs is constructed in the analytic case using Taylor's theorem and corresponding approximation bounds are provided. Finally, it is shown as a corollary of these results that any fading memory filter can be uniformly approximated by a finite Volterra series with finite memory.