# Bootstrapped Model Predictive Control

arXiv: https://arxiv.org/abs/2503.18871 · HTML: https://mlsanity.com/paper/2503.18871/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 36 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Builds on (most cited first, 30 of 30)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- [Mastering Atari Games with Limited Data](https://mlsanity.com/paper/2111.00210/) (2021)
- [Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents](https://mlsanity.com/paper/2408.07199/) (2024)
- [Residual Policy Learning](https://mlsanity.com/paper/1812.06298/) (2018)
- [Transformer-based World Models Are Happy With 100k Interactions](https://mlsanity.com/paper/2303.07109/) (2023)
- [Model Predictive Path Integral Control using Covariance Variable Importance Sampling](https://mlsanity.com/paper/1509.01149/) (2015)
- [Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control](https://mlsanity.com/paper/1811.01848/) (2018)
- [HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation](https://mlsanity.com/paper/2403.10506/) (2024)
- [Motion Policy Networks](https://mlsanity.com/paper/2210.12209/) (2022)
- [Learning Off-Policy with Online Planning](https://mlsanity.com/paper/2008.10066/) (2020)
- [Learning Deep Control Policies for Autonomous Aerial Vehicles with MPC-Guided Policy Search](https://mlsanity.com/paper/1509.06791/) (2015)
- [EfficientZero V2: Mastering Discrete and Continuous Control with Limited Data](https://mlsanity.com/paper/2403.00564/) (2024)
- [Blending MPC & Value Function Approximation for Efficient Reinforcement Learning](https://mlsanity.com/paper/2012.05909/) (2020)
- [DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction](https://mlsanity.com/paper/2203.00494/) (2022)
- [A Generalist Dynamics Model for Control](https://mlsanity.com/paper/2305.10912/) (2023)
- [A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learning](https://mlsanity.com/paper/1707.02515/) (2017)
- [Variational Inference MPC using Normalizing Flows and Out-of-Distribution Projection](https://mlsanity.com/paper/2205.04667/) (2022)
- [Learning to Optimize in Model Predictive Control](https://mlsanity.com/paper/2212.02603/) (2022)
- [Learning Sampling Distributions for Model Predictive Control](https://mlsanity.com/paper/2212.02587/) (2022)
- [Deep Model Predictive Optimization](https://mlsanity.com/paper/2310.04590/) (2023)

## Built on by (most cited first, 16 of 16)

- [A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search](https://mlsanity.com/paper/2506.05294/) (2025)
- [Bootstrap Off-policy with World Model](https://mlsanity.com/paper/2511.00423/) (2025)
- [Generative Models From and For Sampling-Based MPC: A Bootstrapped Approach For Adaptive Contact-Rich Manipulation](https://mlsanity.com/paper/2510.14643/) (2025)
- [A KL-regularization Framework for Learning to Plan with Adaptive Priors](https://mlsanity.com/paper/2510.04280/) (2025)
- [HaM-World: Soft-Hamiltonian World Models with Selective Memory for Planning](https://mlsanity.com/paper/2605.05951/) (2026)
- [EfficientTDMPC: Improved MPC Objectives for Sample-Efficient Continuous Control](https://mlsanity.com/paper/2605.16692/) (2026)
- [SLOPE: Optimistic Potential Landscape Shaping for Model-based Reinforcement Learning](https://mlsanity.com/paper/2602.03201/) (2026)
- [ELVIS: Ensemble-Calibrated Latent Imagination for Long-Horizon Visual MPC](https://mlsanity.com/paper/2605.04709/) (2026)
- [MODIP: Efficient Model-Based Optimization for Diffusion Policies](https://mlsanity.com/paper/2606.10825/) (2026)
- [CAST: Alternating State-Value Targets and Expanded Policy Gradients for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2609.08853/) (2026)
- [Scaling Tasks, Not Samples: Mastering Humanoid Control through Multi-Task Model-Based Reinforcement Learning](https://mlsanity.com/paper/2603.01452/) (2026)
- [Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization](https://mlsanity.com/paper/2605.26282/) (2026)
- [GEM-MPC: Balancing Exploration and Exploitation through Expert-Guided Planning](https://mlsanity.com/paper/2609.21735/) (2026)
- [Think Fast, Plan Selectively: Adaptive Deliberation for Efficient Data-Driven MPC](https://mlsanity.com/paper/2609.32591/) (2026)
- [Chunk-wise Model-based Policy Optimization](https://mlsanity.com/paper/2610.14564/) (2026)
- [Model-Based Online Decision Making via Generative Trajectory Planning](https://mlsanity.com/paper/2610.18677/) (2026)

## Abstract (arXiv)

Model Predictive Control (MPC) has been demonstrated to be effective in continuous control tasks. When a world model and a value function are available, planning a sequence of actions ahead of time leads to a better policy. Existing methods typically obtain the value function and the corresponding policy in a model-free manner. However, we find that such an approach struggles with complex tasks, resulting in poor policy learning and inaccurate value estimation. To address this problem, we leverage the strengths of MPC itself. In this work, we introduce Bootstrapped Model Predictive Control (BMPC), a novel algorithm that performs policy learning in a bootstrapped manner. BMPC learns a network policy by imitating an MPC expert, and in turn, uses this policy to guide the MPC process. Combined with model-based TD-learning, our policy learning yields better value estimation and further boosts the efficiency of MPC. We also introduce a lazy reanalyze mechanism, which enables computationally efficient imitation learning. Our method achieves superior performance over prior works on diverse continuous control tasks. In particular, on challenging high-dimensional locomotion tasks, BMPC significantly improves data efficiency while also enhancing asymptotic performance and training stability, with comparable training time and smaller network sizes. Code is available at https://github.com/wertyuilife2/bmpc.