# Contracting Implicit Recurrent Neural Networks: Stable Models with Improved Trainability

arXiv: https://arxiv.org/abs/1912.10402 · HTML: https://mlsanity.com/paper/1912.10402/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2019.

## Built on by (most cited first, 17 of 17)

- [JFB: Jacobian-Free Backpropagation for Implicit Networks](https://mlsanity.com/paper/2103.12803/) (2021)
- [Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustness](https://mlsanity.com/paper/2104.05942/) (2021)
- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear System](https://mlsanity.com/paper/2112.04219/) (2021)
- [Robust Classification using Contractive Hamiltonian Neural ODEs](https://mlsanity.com/paper/2203.11805/) (2022)
- [RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networks](https://mlsanity.com/paper/2106.08928/) (2021)
- [SDYN-GANs: Adversarial Learning Methods for Multistep Generative Models for General Order Stochastic Dynamics](https://mlsanity.com/paper/2302.03663/) (2023)
- [A Contraction Theory Approach to Optimization Algorithms from Acceleration Flows](https://mlsanity.com/paper/2105.08832/) (2021)
- [Learning of Hamiltonian Dynamics with Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/2312.09734/) (2023)
- [Learning Hamiltonian Dynamics with Reproducing Kernel Hilbert Spaces and Random Features](https://mlsanity.com/paper/2404.07703/) (2024)
- [Learning Stable and Passive Neural Differential Equations](https://mlsanity.com/paper/2404.12554/) (2024)
- [Accelerating Dynamical System Simulations with Contracting and Physics-Projected Neural-Newton Solvers](https://mlsanity.com/paper/2106.02543/) (2021)
- [Dissipative Imitation Learning for Robust Dynamic Output Feedback](https://mlsanity.com/paper/2210.00979/) (2022)
- [Distributed Identification of Contracting and/or Monotone Network Dynamics](https://mlsanity.com/paper/2107.14309/) (2021)
- [Dissipative Imitation Learning for Discrete Dynamic Output Feedback Control with Sparse Data Sets](https://mlsanity.com/paper/2309.06658/) (2023)
- [Robust Convolution Neural ODEs via Contractivity-promoting regularization](https://mlsanity.com/paper/2508.11432/) (2025)

## Abstract (arXiv)

Stability of recurrent models is closely linked with trainability, generalizability and in some applications, safety. Methods that train stable recurrent neural networks, however, do so at a significant cost to expressibility. We propose an implicit model structure that allows for a convex parametrization of stable models using contraction analysis of non-linear systems. Using these stability conditions we propose a new approach to model initialization and then provide a number of empirical results comparing the performance of our proposed model set to previous stable RNNs and vanilla RNNs. By carefully controlling stability in the model, we observe a significant increase in the speed of training and model performance.