# Warm-Starting Collision-Free Model Predictive Control With Object-Centric Diffusion

arXiv: https://arxiv.org/abs/2601.02873 · HTML: https://mlsanity.com/paper/2601.02873/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 26 of its 57 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [Diffusion-based Generation, Optimization, and Planning in 3D Scenes](https://mlsanity.com/paper/2301.06015/) (2023)
- [Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models](https://mlsanity.com/paper/2308.01557/) (2023)
- [Learning Sampling Distributions for Robot Motion Planning](https://mlsanity.com/paper/1709.05448/) (2017)
- [Motion Planning Networks](https://mlsanity.com/paper/1806.05767/) (2018)
- [Learning Robotic Manipulation Policies from Point Clouds with Conditional Flow Matching](https://mlsanity.com/paper/2409.07343/) (2024)
- [Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Control](https://mlsanity.com/paper/1909.04947/) (2019)
- [Diffusion Predictive Control with Constraints](https://mlsanity.com/paper/2412.09342/) (2024)
- [Full-Order Sampling-Based MPC for Torque-Level Locomotion Control via Diffusion-Style Annealing](https://mlsanity.com/paper/2409.15610/) (2024)
- [Learning Implicit Priors for Motion Optimization](https://mlsanity.com/paper/2204.05369/) (2022)
- [DiffusionSeeder: Seeding Motion Optimization with Diffusion for Rapid Motion Planning](https://mlsanity.com/paper/2410.16727/) (2024)
- [Learned Critical Probabilistic Roadmaps for Robotic Motion Planning](https://mlsanity.com/paper/1910.03701/) (2019)
- [Collision-Free MPC for Legged Robots in Static and Dynamic Scenes](https://mlsanity.com/paper/2103.13987/) (2021)
- [PRESTO: Fast Motion Planning Using Diffusion Models Based on Key-Configuration Environment Representation](https://mlsanity.com/paper/2409.16012/) (2024)
- [Toward Near-Globally Optimal Nonlinear Model Predictive Control via Diffusion Models](https://mlsanity.com/paper/2412.08278/) (2024)
- [Object-Centric Representations Improve Policy Generalization in Robot Manipulation](https://mlsanity.com/paper/2505.11563/) (2025)
- [Learning How to Walk: Warm-starting Optimal Control Solver with Memory of Motion](https://mlsanity.com/paper/2001.11751/) (2020)
- [Infinite-Horizon Value Function Approximation for Model Predictive Control](https://mlsanity.com/paper/2502.06760/) (2025)
- [Warm-Starting Optimization-Based Motion Planning for Robotic Manipulators via Point Cloud-Conditioned Flow Matching](https://mlsanity.com/paper/2510.03460/) (2025)
- [Monte Carlo Tree Search with Velocity Obstacles for safe and efficient motion planning in dynamic environments](https://mlsanity.com/paper/2501.09649/) (2025)

## Built on by (most cited first, 1 of 1)

- [RL-Guided PAC-NMPC for Probabilistically-Safe Perception-Based Navigation in Unknown Environments](https://mlsanity.com/paper/2609.39854/) (2026)

## Abstract (arXiv)

Acting in cluttered environments requires predicting and avoiding collisions while still achieving precise control. Conventional optimization-based controllers can enforce physical constraints, but they struggle to produce feasible solutions quickly when many obstacles are present. Diffusion models can generate diverse trajectories around obstacles, yet prior approaches lacked a general and efficient way to condition them on scene structure. In this paper, we show that combining diffusion-based warm-starting conditioned with a latent object-centric representation of the scene and with a collision-aware model predictive controller (MPC) yields reliable and efficient motion generation under strict time limits. Our approach conditions a diffusion transformer on the system state, task, and surroundings, using an object-centric slot attention mechanism to provide a compact obstacle representation suitable for control. The sampled trajectories are refined by an optimal control problem that enforces rigid-body dynamics and signed-distance collision constraints, producing feasible motions in real time. On benchmark tasks, this hybrid method achieved markedly higher success rates and lower latency than sampling-based planners or either component alone. Real-robot experiments with a torque-controlled Panda confirm reliable and safe execution with MPC.