# Dimension reduction in recurrent networks by canonicalization

arXiv: https://arxiv.org/abs/2007.12141 · HTML: https://mlsanity.com/paper/2007.12141/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2020.

## Built on by (most cited first, 11 of 11)

- [Learning strange attractors with reservoir systems](https://mlsanity.com/paper/2108.05024/) (2021)
- [Reservoir kernels and Volterra series](https://mlsanity.com/paper/2212.14641/) (2022)
- [Infinite-dimensional reservoir computing](https://mlsanity.com/paper/2304.00490/) (2023)
- [Memory of recurrent networks: Do we compute it right?](https://mlsanity.com/paper/2305.01457/) (2023)
- [The Gaussian kernel on the circle and spaces that admit isometric embeddings of the circle](https://mlsanity.com/paper/2302.10623/) (2023)
- [Gaussian kernels on non-simply-connected closed Riemannian manifolds are never positive definite](https://mlsanity.com/paper/2303.06558/) (2023)
- [Stochastic dynamics learning with state-space systems](https://mlsanity.com/paper/2508.07876/) (2025)
- [Fading memory and the convolution theorem](https://mlsanity.com/paper/2408.07386/) (2024)
- [Interpretable Design of Reservoir Computing Networks using Realization Theory](https://mlsanity.com/paper/2112.06891/) (2021)
- [Infinite-dimensional Mahalanobis Distance with Applications to Kernelized Novelty Detection](https://mlsanity.com/paper/2407.11873/) (2024)
- [Data-Specific Hyper-Parameter Design: A Paradigm Shift in Reservoir Computing](https://mlsanity.com/paper/2605.25221/) (2026)

## Abstract (arXiv)

Many recurrent neural network machine learning paradigms can be formulated using state-space representations. The classical notion of canonical state-space realization is adapted in this paper to accommodate semi-infinite inputs so that it can be used as a dimension reduction tool in the recurrent networks setup. The so-called input forgetting property is identified as the key hypothesis that guarantees the existence and uniqueness (up to system isomorphisms) of canonical realizations for causal and time-invariant input/output systems with semi-infinite inputs. Additionally, the notion of optimal reduction coming from the theory of symmetric Hamiltonian systems is implemented in our setup to construct canonical realizations out of input forgetting but not necessarily canonical ones. These two procedures are studied in detail in the framework of linear fading memory input/output systems. Finally, the notion of implicit reduction using reproducing kernel Hilbert spaces (RKHS) is introduced which allows, for systems with linear readouts, to achieve dimension reduction without the need to actually compute the reduced spaces introduced in the first part of the paper.