mlsanityData to 14 Oct 2026

DiffDef: A Diffusion Model for Generating Multimodal Goal Shapes From Demonstrations for Deformable Object Manipulation

arXiv 2506.18779 ↗First posted 23 Jun 2025cs.ROcited by 0

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Deformable object manipulation is a key capability in many robotic applications. A promising paradigm for this problem is shape servoing, which aims to control deformable objects toward desired goal shapes. However, existing approaches typically rely on impractical goal-shape acquisition methods, such as domain-knowledge engineering or manual manipulation. Moreover, prior methods generally assume a single deterministic goal and fail to handle multimodal goal settings, a common scenario in many real-world tasks where multiple distinct goal shapes can all lead to successful task completion. In this paper, we introduce DiffDef, a novel neural network that uses a diffusion model to learn a distribution of feasible goal shapes rather than predicting a single deterministic outcome. This allows DiffDef to generate diverse goal configurations while avoiding the mode-averaging artifacts common in deterministic predictors. We evaluate our method on several deformable manipulation tasks inspired by manufacturing and surgical applications, both in simulation and on two physical robotic platforms: the da Vinci Research Kit (dVRK) and a bimanual KUKA-based robotic system. The results demonstrate that DiffDef effectively captures multimodal goal distributions and significantly improves task performance in practical robotic settings. Website: sites.google.com/view/diffdef.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 27 of its 37 references that are in the map. First posted on arXiv 23 Jun 2025.

Builds on

27 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 26

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  4. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 735
  5. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 657
  6. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  7. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 242
  8. 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
  9. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 138
  10. 2019Self-supervised 6D Object Pose Estimation for Robot Manipulationcited by 101
  11. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 90
  12. 2019Learning to Manipulate Deformable Objects without Demonstrationscited by 77
14 more
  1. 20196-DOF Grasping for Target-driven Object Manipulation in Cluttercited by 70
  2. 2019C-Flow: Conditional Generative Flow Models for Images and 3D Point Cloudscited by 50
  3. 2021Challenges and Outlook in Robotic Manipulation of Deformable Objectscited by 43
  4. 2019Learning Continuous 3D Reconstructions for Geometrically Aware Graspingcited by 29
  5. 2023Bridging Language and Action: A Survey of Language-Conditioned Robot Manipulationcited by 22
  6. 2020Multi-Fingered Active Grasp Learningcited by 13
  7. 2020A Progressive Conditional Generative Adversarial Network for Generating Dense and Colored 3D Point Cloudscited by 10
  8. 2021Learning Visual Shape Control of Novel 3D Deformable Objects from Partial-View Point Cloudscited by 9
  9. 2020Vision-based Manipulation of Deformable and Rigid Objects Using Subspace Projections of 2D Contourscited by 5
  10. 2022Lattice-based shape tracking and servoing of elastic objectscited by 5
  11. 2023DeformerNet: Learning Bimanual Manipulation of 3D Deformable Objectscited by 4
  12. 2022Model Predictive Manipulation of Compliant Objects with Multi-Objective Optimizer and Adversarial Network for Occlusion Compensationcited by 3
  13. 2023DefGoalNet: Contextual Goal Learning from Demonstrations For Deformable Object Manipulationcited by 3
  14. 2023Stein Variational Belief Propagation for Multi-Robot Coordinationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024SculptDiff: Learning Robotic Clay Sculpting from Humans with Goal Conditioned Diffusion Policy
  2. 2022Learning Deformable Object Manipulation from Expert Demonstrations
  3. 2026A Perspective on Open Challenges in Deformable Object Manipulation ↗
  4. 2025MoDeSuite: Robot Learning Task Suite for Benchmarking Mobile Manipulation with Deformable Objects
  5. 2024Offline Goal-Conditioned Reinforcement Learning for Shape Control of Deformable Linear Objects
  6. 2024DeformPAM: Data-Efficient Learning for Long-horizon Deformable Object Manipulation via Preference-based Action Alignment
  7. 2025D2PPO: Diffusion Policy Policy Optimization with Dispersive Loss
  8. 2024Planning and Reasoning with 3D Deformable Objects for Hierarchical Text-to-3D Robotic Shaping