# Diffusion Model Predictive Control

arXiv: https://arxiv.org/abs/2410.05364 · HTML: https://mlsanity.com/paper/2410.05364/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 46 of its 57 references that are in the map.
- First posted on arXiv 7 Oct 2024.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 30 of 46)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Offline Reinforcement Learning as One Big Sequence Modeling Problem](https://mlsanity.com/paper/2106.02039/) (2021)
- [MOPO: Model-based Offline Policy Optimization](https://mlsanity.com/paper/2005.13239/) (2020)
- [MOReL : Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2005.05951/) (2020)
- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023)
- [Improved Techniques for Training Consistency Models](https://mlsanity.com/paper/2310.14189/) (2023)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [COMBO: Conservative Offline Model-Based Policy Optimization](https://mlsanity.com/paper/2102.08363/) (2021)
- [Diffusion for World Modeling: Visual Details Matter in Atari](https://mlsanity.com/paper/2405.12399/) (2024)
- [Imitating Human Behaviour with Diffusion Models](https://mlsanity.com/paper/2301.10677/) (2023)
- [Model-Based Offline Planning](https://mlsanity.com/paper/2008.05556/) (2020)
- [CORL: Research-oriented Deep Offline Reinforcement Learning Library](https://mlsanity.com/paper/2210.07105/) (2022)
- [RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2204.12581/) (2022)

## Built on by (most cited first, 30 of 36)

- [On the Guidance of Flow Matching](https://mlsanity.com/paper/2502.02150/) (2025)
- [What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?](https://mlsanity.com/paper/2512.24497/) (2025)
- [Diffusion Predictive Control with Constraints](https://mlsanity.com/paper/2412.09342/) (2024)
- [Monte Carlo Tree Diffusion for System 2 Planning](https://mlsanity.com/paper/2502.07202/) (2025)
- [PoE-World: Compositional World Modeling with Products of Programmatic Experts](https://mlsanity.com/paper/2505.10819/) (2025)
- [DDAT: Diffusion Policies Enforcing Dynamically Admissible Robot Trajectories](https://mlsanity.com/paper/2502.15043/) (2025)
- [A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures](https://mlsanity.com/paper/2602.03604/) (2026)
- [Scalable Offline Model-Based RL with Action Chunks](https://mlsanity.com/paper/2512.08108/) (2025)
- [Adaptive Inference-Time Scaling via Cyclic Diffusion Search](https://mlsanity.com/paper/2505.14036/) (2025)
- [The Duality of Generative AI and Reinforcement Learning in Robotics: A Review](https://mlsanity.com/paper/2410.16411/) (2024)
- [Zero-Shot Trajectory Planning for Signal Temporal Logic Tasks](https://mlsanity.com/paper/2501.13457/) (2025)
- [Fast Monte Carlo Tree Diffusion: 100x Speedup via Parallel Sparse Planning](https://mlsanity.com/paper/2506.09498/) (2025)
- [Dynamics-aware Diffusion Models for Planning and Control](https://mlsanity.com/paper/2504.00236/) (2025)
- [Tree-Guided Diffusion Planner](https://mlsanity.com/paper/2508.21800/) (2025)
- [Generative AI for Autonomous Driving: A Review](https://mlsanity.com/paper/2505.15863/) (2025)
- [Generative Models From and For Sampling-Based MPC: A Bootstrapped Approach For Adaptive Contact-Rich Manipulation](https://mlsanity.com/paper/2510.14643/) (2025)
- [SAD-Flower: Flow Matching for Safe, Admissible, and Dynamically Consistent Planning](https://mlsanity.com/paper/2511.05355/) (2025)
- [Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making](https://mlsanity.com/paper/2512.08280/) (2025)
- [Generative Control as Optimization: Time Unconditional Flow Matching for Adaptive and Robust Robotic Control](https://mlsanity.com/paper/2603.17834/) (2026)
- [Controllable Motion Generation via Diffusion Modal Coupling](https://mlsanity.com/paper/2503.02353/) (2025)
- [Diffusion-Based Forecasting for Uncertainty-Aware Model Predictive Control](https://mlsanity.com/paper/2503.15095/) (2025)
- [A KL-regularization Framework for Learning to Plan with Adaptive Priors](https://mlsanity.com/paper/2510.04280/) (2025)
- [Compositional Monte Carlo Tree Diffusion for Extendable Planning](https://mlsanity.com/paper/2510.21361/) (2025)
- [Flexible Locomotion Learning with Diffusion Model Predictive Control](https://mlsanity.com/paper/2510.04234/) (2025)
- [Long-Horizon Model-Based Offline Reinforcement Learning Without Explicit Conservatism](https://mlsanity.com/paper/2512.04341/) (2025)
- [Accelerated Sequential Flow Matching: A Bayesian Filtering Perspective](https://mlsanity.com/paper/2602.05319/) (2026)
- [Amortizing Trajectory Diffusion with Keyed Drift Fields](https://mlsanity.com/paper/2603.14056/) (2026)
- [Inference Time Policy Optimization for Offline RL with Differentiable World Models](https://mlsanity.com/paper/2603.22430/) (2026)
- [Flow Motion Policy: Manipulator Motion Planning with Flow Matching Models](https://mlsanity.com/paper/2604.07084/) (2026)
- [CMP: Robust Whole-Body Tracking for Loco-Manipulation via Competence Manifold Projection](https://mlsanity.com/paper/2604.07457/) (2026)

## Abstract (arXiv)

We propose Diffusion Model Predictive Control (D-MPC), a novel MPC approach that learns a multi-step action proposal and a multi-step dynamics model, both using diffusion models, and combines them for use in online MPC. On the popular D4RL benchmark, we show performance that is significantly better than existing model-based offline planning methods using MPC (e.g. MBOP) and competitive with state-of-the-art (SOTA) model-based and model-free reinforcement learning methods. We additionally illustrate D-MPC's ability to optimize novel reward functions at run time and adapt to novel dynamics, and highlight its advantages compared to existing diffusion-based planning baselines.