# Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear System

arXiv: https://arxiv.org/abs/2112.04219 · HTML: https://mlsanity.com/paper/2112.04219/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 56 references that are in the map.
- First posted on arXiv 8 Dec 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Neural Lyapunov Control](https://mlsanity.com/paper/2005.00611/) (2020)
- [Naive Exploration is Optimal for Online LQR](https://mlsanity.com/paper/2001.09576/) (2020)
- [Improper Learning for Non-Stochastic Control](https://mlsanity.com/paper/2001.09254/) (2020)
- [Stable Recurrent Models](https://mlsanity.com/paper/1805.10369/) (2018)
- [Lyapunov-stable neural-network control](https://mlsanity.com/paper/2109.14152/) (2021)
- [The Nonstochastic Control Problem](https://mlsanity.com/paper/1911.12178/) (2019)
- [Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustness](https://mlsanity.com/paper/2104.05942/) (2021)
- [Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllers](https://mlsanity.com/paper/2006.07579/) (2020)
- [Contracting Implicit Recurrent Neural Networks: Stable Models with Improved Trainability](https://mlsanity.com/paper/1912.10402/) (2019)
- [Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems](https://mlsanity.com/paper/2109.03861/) (2021)
- [Youla-REN: Learning Nonlinear Feedback Policies with Robust Stability Guarantees](https://mlsanity.com/paper/2112.01253/) (2021)
- [Convex Parameterizations and Fidelity Bounds for Nonlinear Identification and Reduced-Order Modelling](https://mlsanity.com/paper/1701.06652/) (2017)
- [Learning Deep Energy Shaping Policies for Stability-Guaranteed Manipulation](https://mlsanity.com/paper/2103.16432/) (2021)

## Built on by (most cited first, 14 of 14)

- [Neural System Level Synthesis: Learning over All Stabilizing Policies for Nonlinear Systems](https://mlsanity.com/paper/2203.11812/) (2022)
- [Synthesis of Stabilizing Recurrent Equilibrium Network Controllers](https://mlsanity.com/paper/2204.00122/) (2022)
- [Learning to Boost the Performance of Stable Nonlinear Systems](https://mlsanity.com/paper/2405.00871/) (2024)
- [Learning Over Contracting and Lipschitz Closed-Loops for Partially-Observed Nonlinear Systems (Extended Version)](https://mlsanity.com/paper/2304.06193/) (2023)
- [Synthesizing Neural Network Controllers with Closed-Loop Dissipativity Guarantees](https://mlsanity.com/paper/2404.07373/) (2024)
- [Stabilizing reinforcement learning control: A modular framework for optimizing over all stable behavior](https://mlsanity.com/paper/2310.14098/) (2023)
- [Quantization Robustness of Monotone Operator Equilibrium Networks](https://mlsanity.com/paper/2603.10562/) (2026)
- [R2DN: Scalable Parameterization of Contracting and Lipschitz Recurrent Deep Networks](https://mlsanity.com/paper/2504.01250/) (2025)
- [A modular framework for stabilizing deep reinforcement learning control](https://mlsanity.com/paper/2304.03422/) (2023)
- [RobustNeuralNetworks.jl: a Package for Machine Learning and Data-Driven Control with Certified Robustness](https://mlsanity.com/paper/2306.12612/) (2023)
- [MAD: A Magnitude And Direction Policy Parametrization for Stability Constrained Reinforcement Learning](https://mlsanity.com/paper/2504.02565/) (2025)
- [React to Surprises: Stable-by-Design Neural Feedback Control and the Youla-REN](https://mlsanity.com/paper/2506.01226/) (2025)
- [Multitask LQG Control: Performance and Generalization Bounds](https://mlsanity.com/paper/2604.16730/) (2026)
- [Distributed Control of Network Systems in the Space of Stabilizing Graph Neural Network Policies](https://mlsanity.com/paper/2512.18540/) (2025)

## Abstract (arXiv)

This paper proposes a nonlinear policy architecture for control of partially-observed linear dynamical systems providing built-in closed-loop stability guarantees. The policy is based on a nonlinear version of the Youla parameterization, and augments a known stabilizing linear controller with a nonlinear operator from a recently developed class of dynamic neural network models called the recurrent equilibrium network (REN). We prove that RENs are universal approximators of contracting and Lipschitz nonlinear systems, and subsequently show that the the proposed Youla-REN architecture is a universal approximator of stabilizing nonlinear controllers. The REN architecture simplifies learning since unconstrained optimization can be applied, and we consider both a model-based case where exact gradients are available and reinforcement learning using random search with zeroth-order oracles. In simulation examples our method converges faster to better controllers and is more scalable than existing methods, while guaranteeing stability during learning transients.