R2DN: Scalable Parameterization of Contracting and Lipschitz Recurrent Deep Networks
Not on the map. Why?
Abstract (arXiv)
This paper presents the Robust Recurrent Deep Network (R2DN), a scalable parameterization of stable and robust recurrent neural networks for machine learning and data-driven control. We construct R2DNs as the feedback interconnection of a linear time-invariant system and a 1-Lipschitz deep feedforward network, and directly parameterize the weights so that our models are stable (contracting) and robust to input perturbations (Lipschitz) by design. Our parameterization uses a structure similar to the recurrent equilibrium network (REN), but without having to iteratively solve an equilibrium layer at each time-step. This speeds up model inference and training on GPUs, and makes it computationally feasible to scale up the network size and input sequence length in comparison to RENs. We compare R2DNs to RENs on representative problems in nonlinear system identification, observer design, learning-based feedback control, and sequential image classification. We find that training and inference are up to an order of magnitude faster with similar performance, and that they scale more favorably with respect to model expressivity.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 41 of its 69 references that are in the map. First posted on arXiv 1 Apr 2025.
Builds on
41 of 69 references.
References · 41
- 2017Attention Is All You Needcited by 60,361
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2013Intriguing properties of neural networkscited by 5,155
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
- 2022Simplified State Space Layers for Sequence Modelingcited by 788
- 2019Deep Equilibrium Modelscited by 615
- 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 608
- 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 408
- 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
- 2017Adversarial Attacks on Neural Network Policiescited by 320
29 more
- 2018Audio Adversarial Examples: Targeted Attacks on Speech-to-Textcited by 274
- 2015A Simple Way to Initialize Recurrent Networks of Rectified Linear Unitscited by 130
- 2019Implicit Deep Learningcited by 124
- 2019AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networkscited by 112
- 2020Learning Stable Deep Dynamics Modelscited by 111
- 2021Orthogonalizing Convolutional Layers with the Cayley Transformcited by 105
- 2020Monotone operator equilibrium networkscited by 100
- 2021Skew Orthogonal Convolutionscited by 67
- 2023A Unified Algebraic Perspective on Lipschitz Neural Networkscited by 60
- 2023Direct Parameterization of Lipschitz-Bounded Deep Networkscited by 51
- 2020Lipschitz Bounded Equilibrium Networkscited by 49
- 2021Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustnesscited by 45
- 2022Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networkscited by 39
- 2024Towards Scalable and Stable Parallelization of Nonlinear RNNscited by 38
- 2023LipsFormer: Introducing Lipschitz Continuity to Vision Transformerscited by 34
- 2022Training Efficient Controllers via Analytic Policy Gradientcited by 22
- 2021Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear Systemcited by 14
- 2024Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllerscited by 14
- 2024Residual Policy Learning for Perceptive Quadruped Control Using Differentiable Simulationcited by 13
- 2024Learning Neural Contracting Dynamics: Extended Linearization and Global Guaranteescited by 11
- 2024Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networkscited by 9
- 2017Convex Parameterizations and Fidelity Bounds for Nonlinear Identification and Reduced-Order Modellingcited by 8
- 2021RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networkscited by 7
- 2024Learning to Boost the Performance of Stable Nonlinear Systemscited by 7
- 2023Learning Over Contracting and Lipschitz Closed-Loops for Partially-Observed Nonlinear Systems (Extended Version)cited by 6
- 2024Contractive Dynamical Imitation Policies for Efficient Out-of-Sample Recoverycited by 5
- 2024On Robust Reinforcement Learning with Lipschitz-Bounded Policy Networkscited by 4
- 2024Learning Stable and Passive Neural Differential Equationscited by 2
- 2024LipKernel: Lipschitz-Bounded Convolutional Neural Networks via Dissipative Layerscited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.