# A modular framework for stabilizing deep reinforcement learning control

arXiv: https://arxiv.org/abs/2304.03422 · HTML: https://mlsanity.com/paper/2304.03422/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 7 Apr 2023.

## Builds on (most cited first, 12 of 12)

- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019)
- [Actor-Critic Reinforcement Learning for Control with Stability Guarantee](https://mlsanity.com/paper/2004.14288/) (2020)
- [Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustness](https://mlsanity.com/paper/2104.05942/) (2021)
- [Stability-certified reinforcement learning: A control-theoretic perspective](https://mlsanity.com/paper/1810.11505/) (2018)
- [Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning](https://mlsanity.com/paper/2111.07171/) (2021)
- [Almost Surely Stable Deep Dynamics](https://mlsanity.com/paper/2103.14722/) (2021)
- [Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear System](https://mlsanity.com/paper/2112.04219/) (2021)
- [A trajectory-based framework for data-driven system analysis and control](https://mlsanity.com/paper/1903.10723/) (2019)
- [Willems' Fundamental Lemma for State-space Systems and its Extension to Multiple Datasets](https://mlsanity.com/paper/2002.01023/) (2020)

## Built on by (most cited first, 1 of 1)

- [Stabilizing reinforcement learning control: A modular framework for optimizing over all stable behavior](https://mlsanity.com/paper/2310.14098/) (2023)

## Abstract (arXiv)

We propose a framework for the design of feedback controllers that combines the optimization-driven and model-free advantages of deep reinforcement learning with the stability guarantees provided by using the Youla-Kucera parameterization to define the search domain. Recent advances in behavioral systems allow us to construct a data-driven internal model; this enables an alternative realization of the Youla-Kucera parameterization based entirely on input-output exploration data. Using a neural network to express a parameterized set of nonlinear stable operators enables seamless integration with standard deep learning libraries. We demonstrate the approach on a realistic simulation of a two-tank system.