# Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustness

arXiv: https://arxiv.org/abs/2104.05942 · HTML: https://mlsanity.com/paper/2104.05942/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 96 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [Regularisation of Neural Networks by Enforcing Lipschitz Continuity](https://mlsanity.com/paper/1804.04368/) (2018)
- [Certified Defenses against Adversarial Examples](https://mlsanity.com/paper/1801.09344/) (2018)
- [Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks](https://mlsanity.com/paper/1906.04893/) (2019)
- [Implicit Deep Learning](https://mlsanity.com/paper/1908.06315/) (2019)
- [Learning Stable Deep Dynamics Models](https://mlsanity.com/paper/2001.06116/) (2020)
- [Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming](https://mlsanity.com/paper/1903.01287/) (2019)
- [Monotone operator equilibrium networks](https://mlsanity.com/paper/2006.08591/) (2020)
- [Training robust neural networks using Lipschitz bounds](https://mlsanity.com/paper/2005.02929/) (2020)
- [Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples](https://mlsanity.com/paper/1803.01128/) (2018)
- [Online Control with Adversarial Disturbances](https://mlsanity.com/paper/1902.08721/) (2019)
- [Improper Learning for Non-Stochastic Control](https://mlsanity.com/paper/2001.09254/) (2020)
- [Stable Recurrent Models](https://mlsanity.com/paper/1805.10369/) (2018)
- [Limitations of the Lipschitz constant as a defense against adversarial examples](https://mlsanity.com/paper/1807.09705/) (2018)
- [Lipschitz Bounded Equilibrium Networks](https://mlsanity.com/paper/2010.01732/) (2020)
- [Don't Unroll Adjoint: Differentiating SSA-Form Programs](https://mlsanity.com/paper/1810.07951/) (2018)
- [Imitation Learning with Stability and Safety Guarantees](https://mlsanity.com/paper/2012.09293/) (2020)
- [L2-Nonexpansive Neural Networks](https://mlsanity.com/paper/1802.07896/) (2018)
- [Optimal Attacks on Reinforcement Learning Policies](https://mlsanity.com/paper/1907.13548/) (2019)
- [Robust Guarantees for Perception-Based Control](https://mlsanity.com/paper/1907.03680/) (2019)
- [Contracting Implicit Recurrent Neural Networks: Stable Models with Improved Trainability](https://mlsanity.com/paper/1912.10402/) (2019)
- [Identification of block-oriented nonlinear systems starting from linear approximations: A survey](https://mlsanity.com/paper/1607.01217/) (2016)
- [Convex Parameterizations and Fidelity Bounds for Nonlinear Identification and Reduced-Order Modelling](https://mlsanity.com/paper/1701.06652/) (2017)
- [An Analysis of the Expressiveness of Deep Neural Network Architectures Based on Their Lipschitz Constants](https://mlsanity.com/paper/1912.11511/) (2019)
- [Reduced-Order Nonlinear Observers via Contraction Analysis and Convex Optimization](https://mlsanity.com/paper/2012.06158/) (2020)

## Built on by (most cited first, 30 of 45)

- [Direct Parameterization of Lipschitz-Bounded Deep Networks](https://mlsanity.com/paper/2301.11526/) (2023)
- [Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear System](https://mlsanity.com/paper/2112.04219/) (2021)
- [Neural System Level Synthesis: Learning over All Stabilizing Policies for Nonlinear Systems](https://mlsanity.com/paper/2203.11812/) (2022)
- [Youla-REN: Learning Nonlinear Feedback Policies with Robust Stability Guarantees](https://mlsanity.com/paper/2112.01253/) (2021)
- [Lipschitz-bounded 1D convolutional neural networks using the Cayley transform and the controllability Gramian](https://mlsanity.com/paper/2303.11835/) (2023)
- [Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networks](https://mlsanity.com/paper/2402.01344/) (2024)
- [Unconstrained Parametrization of Dissipative and Contracting Neural Ordinary Differential Equations](https://mlsanity.com/paper/2304.02976/) (2023)
- [Robustness against Adversarial Attacks in Neural Networks using Incremental Dissipativity](https://mlsanity.com/paper/2111.12906/) (2021)
- [Learning to optimize with convergence guarantees using nonlinear system theory](https://mlsanity.com/paper/2403.09389/) (2024)
- [Learning to Boost the Performance of Stable Nonlinear Systems](https://mlsanity.com/paper/2405.00871/) (2024)
- [A view on learning robust goal-conditioned value functions: Interplay between RL and MPC](https://mlsanity.com/paper/2502.06996/) (2025)
- [Predictability Enables Parallelization of Nonlinear State Space Models](https://mlsanity.com/paper/2508.16817/) (2025)
- [Learning Over Contracting and Lipschitz Closed-Loops for Partially-Observed Nonlinear Systems (Extended Version)](https://mlsanity.com/paper/2304.06193/) (2023)
- [Model order reduction of deep structured state-space models: A system-theoretic approach](https://mlsanity.com/paper/2403.14833/) (2024)
- [Convolutional Neural Networks as 2-D systems](https://mlsanity.com/paper/2303.03042/) (2023)
- [Contractive Dynamical Imitation Policies for Efficient Out-of-Sample Recovery](https://mlsanity.com/paper/2412.07544/) (2024)
- [Stable Linear Subspace Identification: A Machine Learning Approach](https://mlsanity.com/paper/2311.03197/) (2023)
- [Synthesizing Neural Network Controllers with Closed-Loop Dissipativity Guarantees](https://mlsanity.com/paper/2404.07373/) (2024)
- [State space representations of the Roesser type for convolutional layers](https://mlsanity.com/paper/2403.11938/) (2024)
- [Quantization Robustness of Monotone Operator Equilibrium Networks](https://mlsanity.com/paper/2603.10562/) (2026)
- [Learning Stable and Passive Neural Differential Equations](https://mlsanity.com/paper/2404.12554/) (2024)
- [LipKernel: Lipschitz-Bounded Convolutional Neural Networks via Dissipative Layers](https://mlsanity.com/paper/2410.22258/) (2024)
- [R2DN: Scalable Parameterization of Contracting and Lipschitz Recurrent Deep Networks](https://mlsanity.com/paper/2504.01250/) (2025)
- [Learning Stable and Robust Linear Parameter-Varying State-Space Models](https://mlsanity.com/paper/2304.01828/) (2023)
- [A modular framework for stabilizing deep reinforcement learning control](https://mlsanity.com/paper/2304.03422/) (2023)
- [Behavior quantification as the missing link between fields: Tools for digital psychiatry and their role in the future of neurobiology](https://mlsanity.com/paper/2305.15385/) (2023)
- [RobustNeuralNetworks.jl: a Package for Machine Learning and Data-Driven Control with Certified Robustness](https://mlsanity.com/paper/2306.12612/) (2023)
- [Mapping back and forth between model predictive control and neural networks](https://mlsanity.com/paper/2404.12030/) (2024)
- [On Dissipativity of Cross-Entropy Loss in Training ResNets](https://mlsanity.com/paper/2405.19013/) (2024)
- [L2RU: a Structured State Space Model with prescribed L2-bound](https://mlsanity.com/paper/2503.23818/) (2025)

## Abstract (arXiv)

This paper introduces recurrent equilibrium networks (RENs), a new class of nonlinear dynamical models for applications in machine learning, system identification and control. The new model class admits ``built in'' behavioural guarantees of stability and robustness. All models in the proposed class are contracting -- a strong form of nonlinear stability -- and models can satisfy prescribed incremental integral quadratic constraints (IQC), including Lipschitz bounds and incremental passivity. RENs are otherwise very flexible: they can represent all stable linear systems, all previously-known sets of contracting recurrent neural networks and echo state networks, all deep feedforward neural networks, and all stable Wiener/Hammerstein models, and can approximate all fading-memory and contracting nonlinear systems. RENs are parameterized directly by a vector in RN, i.e. stability and robustness are ensured without parameter constraints, which simplifies learning since HLgeneric methods for unconstrained optimization such as stochastic gradient descent and its variants can be used. The performance and robustness of the new model set is evaluated on benchmark nonlinear system identification problems, and the paper also presents applications in data-driven nonlinear observer design and control with stability guarantees.