React to Surprises: Stable-by-Design Neural Feedback Control and the Youla-REN
Not on the map. Why?
Abstract (arXiv)
We study parameterizations of stabilizing nonlinear policies for learning-based control. We propose a structure based on a nonlinear version of the Youla-Kucera parameterization combined with robust neural networks such as the recurrent equilibrium network (REN). The resulting parameterizations are unconstrained, and hence can be searched over with first-order optimization methods, while always ensuring closed-loop stability by construction. We study the combination of (a) nonlinear dynamics, (b) partial observation, and (c) incremental closed-loop stability requirements (contraction and Lipschitzness). We find that for the combination of (c) with either (a) or (b), a contracting and Lipschitz Youla parameter always leads to contracting and Lipschitz closed loops. However, if all three hold, then incremental stability can be lost with exogenous disturbances. Instead, a weaker condition is maintained, which we call d-tube contraction and Lipschitzness. We further obtain converse results showing that the proposed parameterization covers all contracting and Lipschitz closed loops for certain classes of nonlinear systems. Numerical experiments illustrate the utility of our parameterization when learning controllers with built-in stability certificates for: (i) ``economic'' rewards without stabilizing effects; (ii) short training horizons; and (iii) uncertain systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 28 of its 77 references that are in the map. First posted on arXiv 2 Jun 2025.
Builds on
28 of 77 references.
References · 28
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
- 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
- 2017Adversarial Attacks on Neural Network Policiescited by 320
- 2018Residual Reinforcement Learning for Robot Controlcited by 274
- 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
- 2018Residual Policy Learningcited by 164
- 2022Accelerated Policy Learning with Parallel Differentiable Simulationcited by 104
- 2022Do Differentiable Simulators Give Better Policy Gradients?cited by 89
- 2020Improper Learning for Non-Stochastic Controlcited by 83
- 2023Direct Parameterization of Lipschitz-Bounded Deep Networkscited by 51
- 2021Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustnesscited by 45
16 more
- 2021Contraction Theory for Nonlinear Stability Analysis and Learning-based Control: A Tutorial Overviewcited by 45
- 2022Dojo: A Differentiable Physics Engine for Roboticscited by 26
- 2022Training Efficient Controllers via Analytic Policy Gradientcited by 22
- 2021Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear Systemcited by 14
- 2024Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllerscited by 14
- 2024Residual Policy Learning for Perceptive Quadruped Control Using Differentiable Simulationcited by 13
- 2022Neural System Level Synthesis: Learning over All Stabilizing Policies for Nonlinear Systemscited by 11
- 2021Youla-REN: Learning Nonlinear Feedback Policies with Robust Stability Guaranteescited by 10
- 2017Convex Parameterizations and Fidelity Bounds for Nonlinear Identification and Reduced-Order Modellingcited by 8
- 2023Learning Over Contracting and Lipschitz Closed-Loops for Partially-Observed Nonlinear Systems (Extended Version)cited by 6
- 2020A System Level Approach to Discrete-Time Nonlinear Systemscited by 4
- 2018Specialized Interior Point Algorithm for Stable Nonlinear System Identificationcited by 3
- 2024Parametrizations of All Stable Closed-loop Responses: From Theory to Neural Network Control Designcited by 3
- 2025R2DN: Scalable Parameterization of Contracting and Lipschitz Recurrent Deep Networkscited by 2
- 2020Virtual Control Contraction Metrics: Convex Nonlinear Feedback Design via Behavioral Embeddingcited by 1
- 2023RobustNeuralNetworks.jl: a Package for Machine Learning and Data-Driven Control with Certified Robustnesscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.