# Dynamical Systems as Temporal Feature Spaces

arXiv: https://arxiv.org/abs/1907.06382 · HTML: https://mlsanity.com/paper/1907.06382/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2019.

## Built on by (most cited first, 12 of 12)

- [Reservoir kernels and Volterra series](https://mlsanity.com/paper/2212.14641/) (2022)
- [Infinite-dimensional reservoir computing](https://mlsanity.com/paper/2304.00490/) (2023)
- [Stacked Residuals of Dynamic Layers for Time Series Anomaly Detection](https://mlsanity.com/paper/2202.12457/) (2022)
- [Predictive Modeling in the Reservoir Kernel Motif Space](https://mlsanity.com/paper/2405.07045/) (2024)
- [Residual Reservoir Memory Networks](https://mlsanity.com/paper/2508.09925/) (2025)
- [ParalESN: Enabling parallel information processing in Reservoir Computing](https://mlsanity.com/paper/2601.22296/) (2026)
- [Euler State Networks: Non-dissipative Reservoir Computing](https://mlsanity.com/paper/2203.09382/) (2022)
- [Separation capacity of linear reservoirs with random connectivity matrix](https://mlsanity.com/paper/2404.17429/) (2024)
- [Structuring Multiple Simple Cycle Reservoirs with Particle Swarm Optimization](https://mlsanity.com/paper/2504.05347/) (2025)
- [Deep Residual Echo State Networks: exploring residual orthogonal connections in untrained Recurrent Neural Networks](https://mlsanity.com/paper/2508.21172/) (2025)
- [Symplectic Representation of Legendre Dynamics](https://mlsanity.com/paper/2512.19409/) (2025)
- [Factorization Regret mediates compositional generalization in latent space](https://mlsanity.com/paper/2603.27134/) (2026)

## Abstract (arXiv)

Parameterized state space models in the form of recurrent networks are often used in machine learning to learn from data streams exhibiting temporal dependencies. To break the black box nature of such models it is important to understand the dynamical features of the input driving time series that are formed in the state space. We propose a framework for rigorous analysis of such state representations in vanishing memory state space models such as echo state networks (ESN). In particular, we consider the state space a temporal feature space and the readout mapping from the state space a kernel machine operating in that feature space. We show that: (1) The usual ESN strategy of randomly generating input-to-state, as well as state coupling leads to shallow memory time series representations, corresponding to cross-correlation operator with fast exponentially decaying coefficients; (2) Imposing symmetry on dynamic coupling yields a constrained dynamic kernel matching the input time series with straightforward exponentially decaying motifs or exponentially decaying motifs of the highest frequency; (3) Simple cycle high-dimensional reservoir topology specified only through two free parameters can implement deep memory dynamic kernels with a rich variety of matching motifs. We quantify richness of feature representations imposed by dynamic kernels and demonstrate that for dynamic kernel associated with cycle reservoir topology, the kernel richness undergoes a phase transition close to the edge of stability.