# Echoes of the Past: A Unified Perspective on Fading memory and Echo States

arXiv: https://arxiv.org/abs/2508.19145 · HTML: https://mlsanity.com/paper/2508.19145/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 61 references that are in the map.
- First posted on arXiv 26 Aug 2025.

## Builds on (most cited first, 10 of 10)

- [Echo state networks are universal](https://mlsanity.com/paper/1806.00797/) (2018)
- [Physical reservoir computing -- An introductory perspective](https://mlsanity.com/paper/2005.00992/) (2020)
- [Differentiable reservoir computing](https://mlsanity.com/paper/1902.06094/) (2019)
- [Convergence Properties for Discrete-time Nonlinear Systems](https://mlsanity.com/paper/1612.05327/) (2016)
- [Extending echo state property for quantum reservoir computing](https://mlsanity.com/paper/2403.02686/) (2024)
- [Embedding Information onto a Dynamical System](https://mlsanity.com/paper/2105.10766/) (2021)
- [Feedback-driven recurrent quantum neural network universality](https://mlsanity.com/paper/2506.16332/) (2025)
- [State-space systems as dynamic generative models](https://mlsanity.com/paper/2404.08717/) (2024)
- [Stochastic dynamics learning with state-space systems](https://mlsanity.com/paper/2508.07876/) (2025)
- [Fading memory and the convolution theorem](https://mlsanity.com/paper/2408.07386/) (2024)

## Built on by (most cited first, 2 of 2)

- [Symplectic Representation of Legendre Dynamics](https://mlsanity.com/paper/2512.19409/) (2025)
- [Data-Specific Hyper-Parameter Design: A Paradigm Shift in Reservoir Computing](https://mlsanity.com/paper/2605.25221/) (2026)

## Abstract (arXiv)

Recurrent neural networks (RNNs) have become increasingly popular in information processing tasks involving time series and temporal data. A fundamental property of RNNs is their ability to create reliable input/output responses, often linked to how the network handles its memory of the information it processed. Various notions have been proposed to conceptualize the behavior of memory in RNNs, including steady states, echo states, state forgetting, input forgetting, and fading memory. Although these notions are often used interchangeably, their precise relationships remain unclear. This work aims to unify these notions in a common language, derive new implications and equivalences between them, and provide alternative proofs to some existing results. By clarifying the relationships between these concepts, this research contributes to a deeper understanding of RNNs and their temporal information processing capabilities.