# Path classification by stochastic linear recurrent neural networks

arXiv: https://arxiv.org/abs/2108.03090 · HTML: https://mlsanity.com/paper/2108.03090/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 48 references that are in the map.
- First posted on arXiv 6 Aug 2021.

## Builds on (most cited first, 15 of 15)

- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Understanding and Mitigating the Tradeoff Between Robustness and Accuracy](https://mlsanity.com/paper/2002.10716/) (2020)
- [A Primer on the Signature Method in Machine Learning](https://mlsanity.com/paper/1603.03788/) (2016)
- [Kernels for sequentially ordered data](https://mlsanity.com/paper/1601.08169/) (2016)
- [Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series](https://mlsanity.com/paper/2006.12107/) (2020)
- [Analysis of classifiers' robustness to adversarial perturbations](https://mlsanity.com/paper/1502.02590/) (2015)
- [Approximation Bounds for Random Neural Networks and Reservoir Systems](https://mlsanity.com/paper/2002.05933/) (2020)
- [On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysis](https://mlsanity.com/paper/2009.07799/) (2020)
- [Risk bounds for reservoir computing](https://mlsanity.com/paper/1910.13886/) (2019)
- [Sparse arrays of signatures for online character recognition](https://mlsanity.com/paper/1308.0371/) (2013)
- [full-FORCE: A Target-Based Method for Training Recurrent Networks](https://mlsanity.com/paper/1710.03070/) (2017)
- [Understanding Recurrent Neural Networks Using Nonequilibrium Response Theory](https://mlsanity.com/paper/2006.11052/) (2020)
- [Probabilistic supervised learning](https://mlsanity.com/paper/1801.00753/) (2018)
- [Unfolding recurrence by Green's functions for optimized reservoir computing](https://mlsanity.com/paper/2010.06247/) (2020)
- [Nonlinear Independent Component Analysis for Discrete-Time and Continuous-Time Signals](https://mlsanity.com/paper/2102.02876/) (2021)

## Built on by (most cited first, 1 of 1)

- [Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis](https://mlsanity.com/paper/2402.12241/) (2024)

## Abstract (arXiv)

We investigate the functioning of a classifying biological neural network from the perspective of statistical learning theory, modelled, in a simplified setting, as a continuous-time stochastic recurrent neural network (RNN) with identity activation function. In the purely stochastic (robust) regime, we give a generalisation error bound that holds with high probability, thus showing that the empirical risk minimiser is the best-in-class hypothesis. We show that RNNs retain a partial signature of the paths they are fed as the unique information exploited for training and classification tasks. We argue that these RNNs are easy to train and robust and back these observations with numerical experiments on both synthetic and real data. We also exhibit a trade-off phenomenon between accuracy and robustness.