mlsanityData to 7 Oct 2026

Contracting Implicit Recurrent Neural Networks: Stable Models with Improved Trainability

arXiv 1912.10402 ↗First posted 22 Dec 2019cs.LG cs.SY eess.SYcited by 17

Not on the map. Why?

Abstract (arXiv)

Stability of recurrent models is closely linked with trainability, generalizability and in some applications, safety. Methods that train stable recurrent neural networks, however, do so at a significant cost to expressibility. We propose an implicit model structure that allows for a convex parametrization of stable models using contraction analysis of non-linear systems. Using these stability conditions we propose a new approach to model initialization and then provide a number of empirical results comparing the performance of our proposed model set to previous stable RNNs and vanilla RNNs. By carefully controlling stability in the model, we observe a significant increase in the speed of training and model performance.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 7; 2022: 2; 2023: 5; 2024: 2; 2025: 1.

17 papers.

Sort:

By area:

  1. 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 84
  2. 2021Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustnesscited by 43
  3. 2023Deep networks for system identification: a Surveycited by 35
  4. 2023Physics-Informed Machine Learning for Modeling and Control of Dynamical Systemscited by 28
  5. 2021Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear Systemcited by 13
  6. 2022Robust Classification using Contractive Hamiltonian Neural ODEscited by 13
  7. 2021RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networkscited by 7
  8. 2023SDYN-GANs: Adversarial Learning Methods for Multistep Generative Models for General Order Stochastic Dynamicscited by 4
  9. 2021A Contraction Theory Approach to Optimization Algorithms from Acceleration Flowscited by 3
  10. 2023Learning of Hamiltonian Dynamics with Reproducing Kernel Hilbert Spacescited by 2
  11. 2024Learning Hamiltonian Dynamics with Reproducing Kernel Hilbert Spaces and Random Featurescited by 2
  12. 2024Learning Stable and Passive Neural Differential Equationscited by 2
5 more
  1. 2021Accelerating Dynamical System Simulations with Contracting and Physics-Projected Neural-Newton Solverscited by 1
  2. 2022Dissipative Imitation Learning for Robust Dynamic Output Feedbackcited by 1
  3. 2021Distributed Identification of Contracting and/or Monotone Network Dynamicscited by 0
  4. 2023Dissipative Imitation Learning for Discrete Dynamic Output Feedback Control with Sparse Data Setscited by 0
  5. 2025Robust Convolution Neural ODEs via Contractivity-promoting regularizationcited by 0