# Learning Recurrent Neural Net Models of Nonlinear Systems

arXiv: https://arxiv.org/abs/2011.09573 · HTML: https://mlsanity.com/paper/2011.09573/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2020.

## Built on by (most cited first, 11 of 11)

- [State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memory](https://mlsanity.com/paper/2309.13414/) (2023)
- [Inverse Approximation Theory for Nonlinear Recurrent Neural Networks](https://mlsanity.com/paper/2305.19190/) (2023)
- [PAC-Bayesian bounds for learning LTI-ss systems with input from empirical loss](https://mlsanity.com/paper/2303.16816/) (2023)
- [Length independent generalization bounds for deep SSM architectures via Rademacher contraction and stability constraints](https://mlsanity.com/paper/2405.20278/) (2024)
- [Realization Theory Of Recurrent Neural ODEs Using Polynomial System Embeddings](https://mlsanity.com/paper/2205.11989/) (2022)
- [Universal Time-Uniform Trajectory Approximation for Random Dynamical Systems with Recurrent Neural Networks](https://mlsanity.com/paper/2211.08018/) (2022)
- [PAC-Bayesian-Like Error Bound for a Class of Linear Time-Invariant Stochastic State-Space Models](https://mlsanity.com/paper/2212.14838/) (2022)
- [Upper Approximation Bounds for Neural Oscillators](https://mlsanity.com/paper/2512.01015/) (2025)
- [PAC bounds of continuous Linear Parameter-Varying systems related to neural ODEs](https://mlsanity.com/paper/2307.03630/) (2023)
- [A finite-sample generalization bound for stable LPV systems](https://mlsanity.com/paper/2405.10054/) (2024)
- [Upper Generalization Bounds for Neural Oscillators](https://mlsanity.com/paper/2603.09742/) (2026)

## Abstract (arXiv)

We consider the following learning problem: Given sample pairs of input and output signals generated by an unknown nonlinear system (which is not assumed to be causal or time-invariant), we wish to find a continuous-time recurrent neural net with hyperbolic tangent activation function that approximately reproduces the underlying i/o behavior with high confidence. Leveraging earlier work concerned with matching output derivatives up to a given finite order, we reformulate the learning problem in familiar system-theoretic language and derive quantitative guarantees on the sup-norm risk of the learned model in terms of the number of neurons, the sample size, the number of derivatives being matched, and the regularity properties of the inputs, the outputs, and the unknown i/o map.