# Optimization of the Model Predictive Control Meta-Parameters Through Reinforcement Learning

arXiv: https://arxiv.org/abs/2111.04146 · HTML: https://mlsanity.com/paper/2111.04146/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 49 references that are in the map.
- First posted on arXiv 7 Nov 2021.

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Differentiable MPC for End-to-end Planning and Control](https://mlsanity.com/paper/1810.13400/) (2018)
- [A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems](https://mlsanity.com/paper/1705.01292/) (2017)
- [Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control](https://mlsanity.com/paper/1811.01848/) (2018)
- [Data-driven Economic NMPC using Reinforcement Learning](https://mlsanity.com/paper/1904.04152/) (2019)
- [Safe Reinforcement Learning via Projection on a Safe Set: How to Achieve Optimality?](https://mlsanity.com/paper/2004.00915/) (2020)
- [Performance-oriented model learning for data-driven MPC design](https://mlsanity.com/paper/1904.10839/) (2019)
- [Goal-Driven Dynamics Learning via Bayesian Optimization](https://mlsanity.com/paper/1703.09260/) (2017)
- [An Analysis of Frame-skipping in Reinforcement Learning](https://mlsanity.com/paper/2102.03718/) (2021)
- [Reinforcement Learning of the Prediction Horizon in Model Predictive Control](https://mlsanity.com/paper/2102.11122/) (2021)
- [Reinforcement Learning based on MPC/MHE for Unmodeled and Partially Observable Dynamics](https://mlsanity.com/paper/2103.11871/) (2021)
- [Adaptive Horizon Model Predictive Control](https://mlsanity.com/paper/1602.08619/) (2016)
- [Optimization of the Model Predictive Control Update Interval Using Reinforcement Learning](https://mlsanity.com/paper/2011.13365/) (2020)

## Built on by (most cited first, 7 of 7)

- [A Safe Reinforcement Learning driven Weights-varying Model Predictive Control for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2402.02624/) (2024)
- [Adaptive Stochastic Nonlinear Model Predictive Control with Look-ahead Deep Reinforcement Learning for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2311.04303/) (2023)
- [Suboptimality analysis of receding horizon quadratic control with unknown linear systems and its applications in learning-based control](https://mlsanity.com/paper/2301.07876/) (2023)
- [Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotion](https://mlsanity.com/paper/2307.07752/) (2023)
- [Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agents](https://mlsanity.com/paper/2507.13491/) (2025)
- [A Systematic Review and Taxonomy of Reinforcement Learning-Model Predictive Control Integration for Linear Systems](https://mlsanity.com/paper/2604.21030/) (2026)
- [Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC](https://mlsanity.com/paper/2609.01061/) (2026)

## Abstract (arXiv)

Model predictive control (MPC) is increasingly being considered for control of fast systems and embedded applications. However, the MPC has some significant challenges for such systems. Its high computational complexity results in high power consumption from the control algorithm, which could account for a significant share of the energy resources in battery-powered embedded systems. The MPC parameters must be tuned, which is largely a trial-and-error process that affects the control performance, the robustness and the computational complexity of the controller to a high degree. In this paper, we propose a novel framework in which any parameter of the control algorithm can be jointly tuned using reinforcement learning(RL), with the goal of simultaneously optimizing the control performance and the power usage of the control algorithm. We propose the novel idea of optimizing the meta-parameters of MPCwith RL, i.e. parameters affecting the structure of the MPCproblem as opposed to the solution to a given problem. Our control algorithm is based on an event-triggered MPC where we learn when the MPC should be re-computed, and a dual mode MPC and linear state feedback control law applied in between MPC computations. We formulate a novel mixture-distribution policy and show that with joint optimization we achieve improvements that do not present themselves when optimizing the same parameters in isolation. We demonstrate our framework on the inverted pendulum control task, reducing the total computation time of the control system by 36% while also improving the control performance by 18.4% over the best-performing MPC baseline.