# Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems

arXiv: https://arxiv.org/abs/2109.03861 · HTML: https://mlsanity.com/paper/2109.03861/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 65 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Implicit Deep Learning](https://mlsanity.com/paper/1908.06315/) (2019)
- [Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints](https://mlsanity.com/paper/1408.3595/) (2014)
- [Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming](https://mlsanity.com/paper/1903.01287/) (2019)
- [Memory-based control with recurrent neural networks](https://mlsanity.com/paper/1512.04455/) (2015)
- [Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions](https://mlsanity.com/paper/2004.07584/) (2020)
- [Policy Optimization for H2 Linear Control with H_∞ Robustness Guarantee: Implicit Regularization and Global Convergence](https://mlsanity.com/paper/1910.09496/) (2019)
- [Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllers](https://mlsanity.com/paper/2006.07579/) (2020)
- [Enforcing robust control guarantees within neural network policies](https://mlsanity.com/paper/2011.08105/) (2020)
- [Imitation Learning with Stability and Safety Guarantees](https://mlsanity.com/paper/2012.09293/) (2020)
- [Stability-certified reinforcement learning: A control-theoretic perspective](https://mlsanity.com/paper/1810.11505/) (2018)
- [Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Control](https://mlsanity.com/paper/2102.04393/) (2021)
- [Exploring applications of deep reinforcement learning for real-world autonomous driving systems](https://mlsanity.com/paper/1901.01536/) (2019)
- [From self-tuning regulators to reinforcement learning and back again](https://mlsanity.com/paper/1906.11392/) (2019)
- [Convex Parameterizations and Fidelity Bounds for Nonlinear Identification and Reduced-Order Modelling](https://mlsanity.com/paper/1701.06652/) (2017)
- [H_∞ Model-free Reinforcement Learning with Robust Stability Guarantee](https://mlsanity.com/paper/1911.02875/) (2019)
- [Linear systems with neural network nonlinearities: Improved stability analysis via acausal Zames-Falb multipliers](https://mlsanity.com/paper/2103.17106/) (2021)
- [Exponential Stability Analysis via Integral Quadratic Constraints](https://mlsanity.com/paper/1706.01337/) (2017)
- [Model predictive control for linear uncertain systems using integral quadratic constraints](https://mlsanity.com/paper/2104.05444/) (2021)

## Built on by (most cited first, 16 of 16)

- [Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear System](https://mlsanity.com/paper/2112.04219/) (2021)
- [Neural System Level Synthesis: Learning over All Stabilizing Policies for Nonlinear Systems](https://mlsanity.com/paper/2203.11812/) (2022)
- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [Youla-REN: Learning Nonlinear Feedback Policies with Robust Stability Guarantees](https://mlsanity.com/paper/2112.01253/) (2021)
- [Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulation](https://mlsanity.com/paper/2405.20860/) (2024)
- [Synthesis of Stabilizing Recurrent Equilibrium Network Controllers](https://mlsanity.com/paper/2204.00122/) (2022)
- [Learning to Boost the Performance of Stable Nonlinear Systems](https://mlsanity.com/paper/2405.00871/) (2024)
- [Synthesizing Neural Network Controllers with Closed-Loop Dissipativity Guarantees](https://mlsanity.com/paper/2404.07373/) (2024)
- [Ensuring Both Positivity and Stability Using Sector-Bounded Nonlinearity for Systems with Neural Network Controllers](https://mlsanity.com/paper/2406.12744/) (2024)
- [Stabilizing reinforcement learning control: A modular framework for optimizing over all stable behavior](https://mlsanity.com/paper/2310.14098/) (2023)
- [Preparing for Black Swans: The Antifragility Imperative for Machine Learning](https://mlsanity.com/paper/2405.11397/) (2024)
- [Robust Recurrent Neural Network to Identify Ship Motion in Open Water with Performance Guarantees -- Technical Report](https://mlsanity.com/paper/2212.05781/) (2022)
- [Rational Neural Network Controllers](https://mlsanity.com/paper/2307.06287/) (2023)
- [Stabilization of industrial processes with time series machine learning](https://mlsanity.com/paper/2506.22502/) (2025)
- [MSACL: Multi-Step Actor-Critic Learning with Lyapunov Certificates for Exponentially Stabilizing Control](https://mlsanity.com/paper/2512.24955/) (2025)
- [Learning Provable Neural Network Observer for Uncertain Dynamical Systems](https://mlsanity.com/paper/2609.30819/) (2026)

## Abstract (arXiv)

Neural network controllers have become popular in control tasks thanks to their flexibility and expressivity. Stability is a crucial property for safety-critical dynamical systems, while stabilization of partially observed systems, in many cases, requires controllers to retain and process long-term memories of the past. We consider the important class of recurrent neural networks (RNN) as dynamic controllers for nonlinear uncertain partially-observed systems, and derive convex stability conditions based on integral quadratic constraints, S-lemma and sequential convexification. To ensure stability during the learning and control process, we propose a projected policy gradient method that iteratively enforces the stability conditions in the reparametrized space taking advantage of mild additional information on system dynamics. Numerical experiments show that our method learns stabilizing controllers while using fewer samples and achieving higher final performance compared with policy gradient.