# Practical Reinforcement Learning of Stabilizing Economic MPC

arXiv: https://arxiv.org/abs/1904.04614 · HTML: https://mlsanity.com/paper/1904.04614/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2019.

## Built on by (most cited first, 17 of 17)

- [Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians](https://mlsanity.com/paper/2102.13073/) (2021)
- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [Stability-Constrained Markov Decision Processes Using MPC](https://mlsanity.com/paper/2102.01383/) (2021)
- [Learning Stochastic Parametric Differentiable Predictive Control Policies](https://mlsanity.com/paper/2203.01447/) (2022)
- [Critic as Lyapunov function (CALF): a model-free, stability-ensuring agent](https://mlsanity.com/paper/2409.09869/) (2024)
- [ZipMPC: Compressed Context-Dependent MPC Cost via Imitation Learning](https://mlsanity.com/paper/2507.13088/) (2025)
- [Approximate Robust NMPC using Reinforcement Learning](https://mlsanity.com/paper/2104.02743/) (2021)
- [A Reinforcement Learning-based Economic Model Predictive Control Framework for Autonomous Operation of Chemical Reactors](https://mlsanity.com/paper/2105.02656/) (2021)
- [A note on stabilizing reinforcement learning](https://mlsanity.com/paper/2111.12316/) (2021)
- [Control invariant set enhanced safe reinforcement learning: improved sampling efficiency, guaranteed stability and robustness](https://mlsanity.com/paper/2305.15602/) (2023)
- [Control invariant set enhanced reinforcement learning for process control: improved sampling efficiency and guaranteed stability](https://mlsanity.com/paper/2304.05509/) (2023)
- [A novel agent with formal goal-reaching guarantees: an experimental study with a mobile robot](https://mlsanity.com/paper/2409.14867/) (2024)
- [A framework for online, stabilizing reinforcement learning](https://mlsanity.com/paper/2207.08730/) (2022)
- [Convergent NMPC-based Reinforcement Learning Using Deep Expected Sarsa and Nonlinear Temporal Difference Learning](https://mlsanity.com/paper/2502.04925/) (2025)
- [Human-aligned Safe Reinforcement Learning for Highway On-Ramp Merging in Dense Traffic](https://mlsanity.com/paper/2503.02624/) (2025)
- [Mission-Aligned Learning-Informed Control of Autonomous Systems: Formulation and Foundations](https://mlsanity.com/paper/2507.04356/) (2025)
- [Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC](https://mlsanity.com/paper/2609.01061/) (2026)

## Abstract (arXiv)

Reinforcement Learning (RL) has demonstrated a huge potential in learning optimal policies without any prior knowledge of the process to be controlled. Model Predictive Control (MPC) is a popular control technique which is able to deal with nonlinear dynamics and state and input constraints. The main drawback of MPC is the need of identifying an accurate model, which in many cases cannot be easily obtained. Because of model inaccuracy, MPC can fail at delivering satisfactory closed-loop performance. Using RL to tune the MPC formulation or, conversely, using MPC as a function approximator in RL allows one to combine the advantages of the two techniques. This approach has important advantages, but it requires an adaptation of the existing algorithms. We therefore propose an improved RL algorithm for MPC and test it in simulations on a rather challenging example.