# Learning High-Level Policies for Model Predictive Control

arXiv: https://arxiv.org/abs/2007.10284 · HTML: https://mlsanity.com/paper/2007.10284/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2020.

## Built on by (most cited first, 10 of 10)

- [Policy Search for Model Predictive Control with Application to Agile Drone Flight](https://mlsanity.com/paper/2112.03850/) (2021)
- [RAMP-Net: A Robust Adaptive MPC for Quadrotors via Physics-informed Neural Network](https://mlsanity.com/paper/2209.09025/) (2022)
- [Learning the References of Online Model Predictive Control for Urban Self-Driving](https://mlsanity.com/paper/2308.15808/) (2023)
- [Certificated Actor-Critic: Hierarchical Reinforcement Learning with Control Barrier Functions for Safe Navigation](https://mlsanity.com/paper/2501.17424/) (2025)
- [Latent Linear Quadratic Regulator for Robotic Control Tasks](https://mlsanity.com/paper/2407.11107/) (2024)
- [Non-Equilibrium MAV-Capture-MAV via Time-Optimal Planning and Reinforcement Learning](https://mlsanity.com/paper/2503.06578/) (2025)
- [Incorporating Recurrent Reinforcement Learning into Model Predictive Control for Adaptive Control in Autonomous Driving](https://mlsanity.com/paper/2301.13313/) (2023)
- [Agile Robotics: Optimal Control, Reinforcement Learning, and Differentiable Simulation](https://mlsanity.com/paper/2407.01568/) (2024)
- [Hierarchical Reinforcement Learning with Low-Level MPC for Multi-Agent Control](https://mlsanity.com/paper/2509.15799/) (2025)
- [Hierarchical Decision Making with Structured Policies: A Principled Design via Inverse Optimization](https://mlsanity.com/paper/2606.28764/) (2026)

## Abstract (arXiv)

The combination of policy search and deep neural networks holds the promise of automating a variety of decision-making tasks. Model Predictive Control (MPC) provides robust solutions to robot control tasks by making use of a dynamical model of the system and solving an optimization problem online over a short planning horizon. In this work, we leverage probabilistic decision-making approaches and the generalization capability of artificial neural networks to the powerful online optimization by learning a deep high-level policy for the MPC (High-MPC). Conditioning on robot's local observations, the trained neural network policy is capable of adaptively selecting high-level decision variables for the low-level MPC controller, which then generates optimal control commands for the robot. First, we formulate the search of high-level decision variables for MPC as a policy search problem, specifically, a probabilistic inference problem. The problem can be solved in a closed-form solution. Second, we propose a self-supervised learning algorithm for learning a neural network high-level policy, which is useful for online hyperparameter adaptations in highly dynamic environments. We demonstrate the importance of incorporating the online adaption into autonomous robots by using the proposed method to solve a challenging control problem, where the task is to control a simulated quadrotor to fly through a swinging gate. We show that our approach can handle situations that are difficult for standard MPC.