# A stabilizing reinforcement learning approach for sampled systems with partially unknown models

arXiv: https://arxiv.org/abs/2208.14714 · HTML: https://mlsanity.com/paper/2208.14714/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 78 references that are in the map.
- First posted on arXiv 31 Aug 2022.

## Builds on (most cited first, 15 of 15)

- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Learning-based Model Predictive Control for Safe Exploration](https://mlsanity.com/paper/1803.08287/) (2018)
- [The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems](https://mlsanity.com/paper/1808.00924/) (2018)
- [Stability-certified reinforcement learning: A control-theoretic perspective](https://mlsanity.com/paper/1810.11505/) (2018)
- [Adaptive Safety with Control Barrier Functions](https://mlsanity.com/paper/1910.00555/) (2019)
- [Safe and Fast Tracking on a Robot Manipulator: Robust MPC and Neural Network Control](https://mlsanity.com/paper/1912.10360/) (2019)
- [A computationally efficient robust model predictive control framework for uncertain nonlinear systems -- extended version](https://mlsanity.com/paper/1910.12081/) (2019)
- [Adaptive CLF-MPC With Application To Quadrupedal Robots](https://mlsanity.com/paper/2112.04536/) (2021)
- [Practical sample-and-hold stabilization of nonlinear systems under approximate optimizers](https://mlsanity.com/paper/1803.02203/) (2018)
- [On inf-convolution-based robust practical stabilization under computational uncertainty](https://mlsanity.com/paper/2102.04309/) (2021)
- [On stochastic stabilization via non-smooth control Lyapunov functions](https://mlsanity.com/paper/2205.13409/) (2022)
- [Value and Policy Iteration in Optimal Control and Adaptive Dynamic Programming](https://mlsanity.com/paper/1507.01026/) (2015)
- [Safe Model-Based Reinforcement Learning for Systems with Parametric Uncertainties](https://mlsanity.com/paper/2007.12666/) (2020)
- [Adaptive Variants of Optimal Feedback Policies](https://mlsanity.com/paper/2104.02709/) (2021)

## Abstract (arXiv)

Reinforcement learning is commonly associated with training of reward-maximizing (or cost-minimizing) agents, in other words, controllers. It can be applied in model-free or model-based fashion, using a priori or online collected system data to train involved parametric architectures. In general, online reinforcement learning does not guarantee closed loop stability unless special measures are taken, for instance, through learning constraints or tailored training rules. Particularly promising are hybrids of reinforcement learning with "classical" control approaches. In this work, we suggest a method to guarantee practical stability of the system-controller closed loop in a purely online learning setting, i.e., without offline training. Moreover, we assume only partial knowledge of the system model. To achieve the claimed results, we employ techniques of classical adaptive control. The implementation of the overall control scheme is provided explicitly in a digital, sampled setting. That is, the controller receives the state of the system and computes the control action at discrete, specifically, equidistant moments in time. The method is tested in adaptive traction control and cruise control where it proved to significantly reduce the cost.