# Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation

arXiv: https://arxiv.org/abs/2412.02676 · HTML: https://mlsanity.com/paper/2412.02676/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 44 of its 59 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 30 of 44)

- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022)
- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation](https://mlsanity.com/paper/2401.02117/) (2024)
- [IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies](https://mlsanity.com/paper/2304.10573/) (2023)
- [TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement](https://mlsanity.com/paper/2306.08637/) (2023)
- [Imitating Human Behaviour with Diffusion Models](https://mlsanity.com/paper/2301.10677/) (2023)
- [Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets](https://mlsanity.com/paper/2109.13396/) (2021)
- [Goal-Conditioned Imitation Learning using Score-based Diffusion Policies](https://mlsanity.com/paper/2304.02532/) (2023)
- [RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation](https://mlsanity.com/paper/1811.02790/) (2018)
- [DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality](https://mlsanity.com/paper/2210.13702/) (2022)
- [ACRONYM: A Large-Scale Grasp Dataset Based on Simulation](https://mlsanity.com/paper/2011.09584/) (2020)
- [Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes](https://mlsanity.com/paper/2211.11744/) (2022)
- [General In-Hand Object Rotation with Vision and Touch](https://mlsanity.com/paper/2309.09979/) (2023)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [Imitating Task and Motion Planning with Visuomotor Transformers](https://mlsanity.com/paper/2305.16309/) (2023)
- [DTC: Deep Tracking Control](https://mlsanity.com/paper/2309.15462/) (2023)
- [Motion Policy Networks](https://mlsanity.com/paper/2210.12209/) (2022)
- [ToolFlowNet: Robotic Manipulation with Tools via Predicting Tool Flow from Point Clouds](https://mlsanity.com/paper/2211.09006/) (2022)
- [Global Planning for Contact-Rich Manipulation via Local Smoothing of Quasi-dynamic Contact Models](https://mlsanity.com/paper/2206.10787/) (2022)
- [Stabilize to Act: Learning to Coordinate for Bimanual Manipulation](https://mlsanity.com/paper/2309.01087/) (2023)
- [MPC-Net: A First Principles Guided Policy Search](https://mlsanity.com/paper/1909.05197/) (2019)
- [Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds](https://mlsanity.com/paper/2210.07442/) (2022)
- [Dojo: A Differentiable Physics Engine for Robotics](https://mlsanity.com/paper/2203.00806/) (2022)

## Built on by (most cited first, 8 of 8)

- [Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids](https://mlsanity.com/paper/2502.20396/) (2025)
- [Physics-Driven Data Generation for Contact-Rich Manipulation via Trajectory Optimization](https://mlsanity.com/paper/2502.20382/) (2025)
- [Empirical Analysis of Sim-and-Real Cotraining of Diffusion Policies for Planar Pushing from Pixels](https://mlsanity.com/paper/2503.22634/) (2025)
- [LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrations](https://mlsanity.com/paper/2508.17547/) (2025)
- [Enhancing Diffusion Policy with Classifier-Free Guidance for Temporal Robotic Tasks](https://mlsanity.com/paper/2510.09786/) (2025)
- [Diffusing Trajectory Optimization Problems for Recovery During Multi-Finger Manipulation](https://mlsanity.com/paper/2510.07030/) (2025)
- [DATAFARM: Distribution-Aligned Task and Motion Planning for Fine-Tuning Vision-Language-Action Models](https://mlsanity.com/paper/2609.12316/) (2026)
- [Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation](https://mlsanity.com/paper/2605.28812/) (2026)

## Abstract (arXiv)

Contact-rich bimanual manipulation involves precise coordination of two arms to change object states through strategically selected contacts and motions. Due to the inherent complexity of these tasks, acquiring sufficient demonstration data and training policies that generalize to unseen scenarios remain a largely unresolved challenge. Building on recent advances in planning through contacts, we introduce Generalizable Planning-Guided Diffusion Policy Learning (GLIDE), an approach that effectively learns to solve contact-rich bimanual manipulation tasks by leveraging model-based motion planners to generate demonstration data in high-fidelity physics simulation. Through efficient planning in randomized environments, our approach generates large-scale and high-quality synthetic motion trajectories for tasks involving diverse objects and transformations. We then train a task-conditioned diffusion policy via behavior cloning using these demonstrations. To tackle the sim-to-real gap, we propose a set of essential design options in feature extraction, task representation, action prediction, and data augmentation that enable learning robust prediction of smooth action sequences and generalization to unseen scenarios. Through experiments in both simulation and the real world, we demonstrate that our approach can enable a bimanual robotic system to effectively manipulate objects of diverse geometries, dimensions, and physical properties. Website: https://glide-manip.github.io/