# Data-driven Economic NMPC using Reinforcement Learning

arXiv: https://arxiv.org/abs/1904.04152 · HTML: https://mlsanity.com/paper/1904.04152/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2019.

## Built on by (most cited first, 30 of 68)

- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions](https://mlsanity.com/paper/2104.08171/) (2021)
- [MPC-based Reinforcement Learning for Economic Problems with Application to Battery Storage](https://mlsanity.com/paper/2104.02411/) (2021)
- [Optimization of the Model Predictive Control Meta-Parameters Through Reinforcement Learning](https://mlsanity.com/paper/2111.04146/) (2021)
- [UAV Path Planning Employing MPC- Reinforcement Learning Method Considering Collision Avoidance](https://mlsanity.com/paper/2302.10669/) (2023)
- [Model Predictive Control and Reinforcement Learning: A Unified Framework Based on Dynamic Programming](https://mlsanity.com/paper/2406.00592/) (2024)
- [A view on learning robust goal-conditioned value functions: Interplay between RL and MPC](https://mlsanity.com/paper/2502.06996/) (2025)
- [End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Control](https://mlsanity.com/paper/2308.01674/) (2023)
- [Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems](https://mlsanity.com/paper/2404.15199/) (2024)
- [AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Control](https://mlsanity.com/paper/2406.03995/) (2024)
- [Differentiable Nonlinear Model Predictive Control](https://mlsanity.com/paper/2505.01353/) (2025)
- [Stability-Constrained Markov Decision Processes Using MPC](https://mlsanity.com/paper/2102.01383/) (2021)
- [Regret Analysis of Learning-Based MPC with Partially-Unknown Cost Function](https://mlsanity.com/paper/2108.02307/) (2021)
- [Learning-based MPC from Big Data Using Reinforcement Learning](https://mlsanity.com/paper/2301.01667/) (2023)
- [Evaluation of MPC-based Imitation Learning for Human-like Autonomous Driving](https://mlsanity.com/paper/2211.12111/) (2022)
- [Learning-Aided Warmstart of Model Predictive Control in Uncertain Fast-Changing Traffic](https://mlsanity.com/paper/2310.02918/) (2023)
- [Reinforcement Learning with Model Predictive Control for Highway Ramp Metering](https://mlsanity.com/paper/2311.08820/) (2023)
- [Reinforced Model Predictive Control via Trust-Region Quasi-Newton Policy Optimization](https://mlsanity.com/paper/2405.17983/) (2024)
- [Quasi-Newton Iteration in Deterministic Policy Gradient](https://mlsanity.com/paper/2203.13854/) (2022)
- [MPC-Inspired Reinforcement Learning for Verifiable Model-Free Control](https://mlsanity.com/paper/2312.05332/) (2023)
- [Flipping-based Policy for Chance-Constrained Markov Decision Processes](https://mlsanity.com/paper/2410.06474/) (2024)
- [Probabilistically safe and efficient model-based reinforcement learning](https://mlsanity.com/paper/2504.00626/) (2025)
- [MPCritic: A plug-and-play MPC architecture for reinforcement learning](https://mlsanity.com/paper/2504.01086/) (2025)
- [Generative AI for Autonomous Driving: A Review](https://mlsanity.com/paper/2505.15863/) (2025)
- [ZipMPC: Compressed Context-Dependent MPC Cost via Imitation Learning](https://mlsanity.com/paper/2507.13088/) (2025)
- [Model Predictive Control-Guided Reinforcement Learning for Implicit Balancing](https://mlsanity.com/paper/2510.04868/) (2025)
- [Approximate Robust NMPC using Reinforcement Learning](https://mlsanity.com/paper/2104.02743/) (2021)
- [A Reinforcement Learning-based Economic Model Predictive Control Framework for Autonomous Operation of Chemical Reactors](https://mlsanity.com/paper/2105.02656/) (2021)
- [Proximal Policy Optimization for Tracking Control Exploiting Future Reference Information](https://mlsanity.com/paper/2107.09647/) (2021)

## Abstract (arXiv)

Reinforcement Learning (RL) is a powerful tool to perform data-driven optimal control without relying on a model of the system. However, RL struggles to provide hard guarantees on the behavior of the resulting control scheme. In contrast, Nonlinear Model Predictive Control (NMPC) and Economic NMPC (ENMPC) are standard tools for the closed-loop optimal control of complex systems with constraints and limitations, and benefit from a rich theory to assess their closed-loop behavior. Unfortunately, the performance of (E)NMPC hinges on the quality of the model underlying the control scheme. In this paper, we show that an (E)NMPC scheme can be tuned to deliver the optimal policy of the real system even when using a wrong model. This result also holds for real systems having stochastic dynamics. This entails that ENMPC can be used as a new type of function approximator within RL. Furthermore, we investigate our results in the context of ENMPC and formally connect them to the concept of dissipativity, which is central for the ENMPC stability. Finally, we detail how these results can be used to deploy classic RL tools for tuning (E)NMPC schemes. We apply these tools on both a classical linear MPC setting and a standard nonlinear example from the ENMPC literature.