# Safe Reinforcement Learning Using Robust MPC

arXiv: https://arxiv.org/abs/1906.04005 · HTML: https://mlsanity.com/paper/1906.04005/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2019.

## Built on by (most cited first, 30 of 85)

- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking](https://mlsanity.com/paper/2205.06750/) (2022)
- [Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians](https://mlsanity.com/paper/2102.13073/) (2021)
- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [Safe Chance Constrained Reinforcement Learning for Batch Process Control](https://mlsanity.com/paper/2104.11706/) (2021)
- [Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty](https://mlsanity.com/paper/2210.05989/) (2022)
- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [Reinforcement Learning of the Prediction Horizon in Model Predictive Control](https://mlsanity.com/paper/2102.11122/) (2021)
- [Improve Agents without Retraining: Parallel Tree Search with Off-Policy Correction](https://mlsanity.com/paper/2107.01715/) (2021)
- [Revisiting Implicit Differentiation for Learning Problems in Optimal Control](https://mlsanity.com/paper/2310.14468/) (2023)
- [Stability-informed Bayesian Optimization for MPC Cost Function Learning](https://mlsanity.com/paper/2404.12187/) (2024)
- [Discrete GCBF Proximal Policy Optimization for Multi-agent Safe Optimal Control](https://mlsanity.com/paper/2502.03640/) (2025)
- [Neural Lyapunov Differentiable Predictive Control](https://mlsanity.com/paper/2205.10728/) (2022)
- [Reinforcement Learning-based Receding Horizon Control using Adaptive Control Barrier Functions for Safety-Critical Systems](https://mlsanity.com/paper/2403.17338/) (2024)
- [Learning to Boost the Performance of Stable Nonlinear Systems](https://mlsanity.com/paper/2405.00871/) (2024)
- [Safe and Stable Closed-Loop Learning for Neural-Network-Supported Model Predictive Control](https://mlsanity.com/paper/2409.10171/) (2024)
- [Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Control](https://mlsanity.com/paper/2206.01341/) (2022)
- [Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL](https://mlsanity.com/paper/2504.15425/) (2025)
- [Stability-Constrained Markov Decision Processes Using MPC](https://mlsanity.com/paper/2102.01383/) (2021)
- [Regret Analysis of Learning-Based MPC with Partially-Unknown Cost Function](https://mlsanity.com/paper/2108.02307/) (2021)
- [Guaranteeing Control Requirements via Reward Shaping in Reinforcement Learning](https://mlsanity.com/paper/2311.10026/) (2023)
- [Deployable Reinforcement Learning with Variable Control Rate](https://mlsanity.com/paper/2401.09286/) (2024)
- [Control-Tutored Reinforcement Learning: Towards the Integration of Data-Driven and Model-Based Control](https://mlsanity.com/paper/2112.06018/) (2021)
- [Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles](https://mlsanity.com/paper/2112.11217/) (2021)
- [Learning Stochastic Parametric Differentiable Predictive Control Policies](https://mlsanity.com/paper/2203.01447/) (2022)
- [Reinforcement Learning with Model Predictive Control for Highway Ramp Metering](https://mlsanity.com/paper/2311.08820/) (2023)
- [Optimal Transport-Assisted Risk-Sensitive Q-Learning](https://mlsanity.com/paper/2406.11774/) (2024)
- [Critic as Lyapunov function (CALF): a model-free, stability-ensuring agent](https://mlsanity.com/paper/2409.09869/) (2024)
- [A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions](https://mlsanity.com/paper/2508.09128/) (2025)

## Abstract (arXiv)

Reinforcement Learning (RL) has recently impressed the world with stunning results in various applications. While the potential of RL is now well-established, many critical aspects still need to be tackled, including safety and stability issues. These issues, while partially neglected by the RL community, are central to the control community which has been widely investigating them. Model Predictive Control (MPC) is one of the most successful control techniques because, among others, of its ability to provide such guarantees even for uncertain constrained systems. Since MPC is an optimization-based technique, optimality has also often been claimed. Unfortunately, the performance of MPC is highly dependent on the accuracy of the model used for predictions. In this paper, we propose to combine RL and MPC in order to exploit the advantages of both and, therefore, obtain a controller which is optimal and safe. We illustrate the results with a numerical example in simulations.