# Universality of Real Minimal Complexity Reservoir

arXiv: https://arxiv.org/abs/2408.08071 · HTML: https://mlsanity.com/paper/2408.08071/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 19 references that are in the map.
- First posted on arXiv 15 Aug 2024.

## Builds on (most cited first, 5 of 5)

- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Echo state networks are universal](https://mlsanity.com/paper/1806.00797/) (2018)
- [Reservoir Computing Universality With Stochastic Inputs](https://mlsanity.com/paper/1807.02621/) (2018)
- [Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems](https://mlsanity.com/paper/1712.00754/) (2017)
- [Simple Cycle Reservoirs are Universal](https://mlsanity.com/paper/2308.10793/) (2023)

## Built on by (most cited first, 3 of 3)

- [Minimal Deterministic Echo State Networks Outperform Random Reservoirs in Learning Chaotic Dynamics](https://mlsanity.com/paper/2507.06050/) (2025)
- [Structuring Multiple Simple Cycle Reservoirs with Particle Swarm Optimization](https://mlsanity.com/paper/2504.05347/) (2025)
- [Symplectic Representation of Legendre Dynamics](https://mlsanity.com/paper/2512.19409/) (2025)

## Abstract (arXiv)

Reservoir Computing (RC) models, a subclass of recurrent neural networks, are distinguished by their fixed, non-trainable input layer and dynamically coupled reservoir, with only the static readout layer being trained. This design circumvents the issues associated with backpropagating error signals through time, thereby enhancing both stability and training efficiency. RC models have been successfully applied across a broad range of application domains. Crucially, they have been demonstrated to be universal approximators of time-invariant dynamic filters with fading memory, under various settings of approximation norms and input driving sources. Simple Cycle Reservoirs (SCR) represent a specialized class of RC models with a highly constrained reservoir architecture, characterized by uniform ring connectivity and binary input-to-reservoir weights with an aperiodic sign pattern. For linear reservoirs, given the reservoir size, the reservoir construction has only one degree of freedom -- the reservoir cycle weight. Such architectures are particularly amenable to hardware implementations without significant performance degradation in many practical tasks. In this study we endow these observations with solid theoretical foundations by proving that SCRs operating in real domain are universal approximators of time-invariant dynamic filters with fading memory. Our results supplement recent research showing that SCRs in the complex domain can approximate, to arbitrary precision, any unrestricted linear reservoir with a non-linear readout. We furthermore introduce a novel method to drastically reduce the number of SCR units, making such highly constrained architectures natural candidates for low-complexity hardware implementations. Our findings are supported by empirical studies on real-world time series datasets.