mlsanityData to 7 Oct 2026

Composable Part-Based Manipulation

arXiv 2405.05876 ↗First posted 9 May 2024cs.RO cs.AI cs.CVcited by 17

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

In this paper, we propose composable part-based manipulation (CPM), a novel approach that leverages object-part decomposition and part-part correspondences to improve learning and generalization of robotic manipulation skills. By considering the functional correspondences between object parts, we conceptualize functional actions, such as pouring and constrained placing, as combinations of different correspondence constraints. CPM comprises a collection of composable diffusion models, where each model captures a different inter-object correspondence. These diffusion models can generate parameters for manipulation skills based on the specific object parts. Leveraging part-based correspondences coupled with the task decomposition into distinct constraints enables strong generalization to novel objects and object categories. We validate our approach in both simulated and real-world scenarios, demonstrating its effectiveness in achieving robust and generalized manipulation capabilities.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 53 of its 61 references that are in the map. First posted on arXiv 9 May 2024.

Builds on

53 of 61 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023Learning to Read Braille: Bridging the Tactile Reality Gap with Diffusion Models3 Apr
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  3. 2023ReorientDiff: Diffusion Model based Reorientation for Object Manipulation28 Feb
  4. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenes15 Jan
  5. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov
  6. 2022StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects8 Nov
  7. 2022DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics5 Oct
  8. 2022SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusion8 Sep

Other references · 45

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Point Transformercited by 1,462
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  4. 2020PCT: Point cloud transformercited by 730
  5. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  6. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  7. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  8. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  9. 2020Integrated Task and Motion Planningcited by 281
  10. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 216
  11. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 213
  12. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 204
33 more
  1. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 179
  2. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  3. 2019kPAM: KeyPoint Affordances for Category-Level Robotic Manipulationcited by 152
  4. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 135
  5. 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 135
  6. 2017AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detectioncited by 126
  7. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 117
  8. 2023Collaborative Diffusion for Multi-Modal Face Generation and Editingcited by 116
  9. 2020ACRONYM: A Large-Scale Grasp Dataset Based on Simulationcited by 115
  10. 2022Grounding Language with Visual Affordances over Unstructured Datacited by 101
  11. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  12. 2021VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objectscited by 78
  13. 2020Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learningcited by 69
  14. 2019KETO: Learning Keypoint Representations for Tool Manipulationcited by 68
  15. 2021UMPNet: Universal Manipulation Policy Network for Articulated Objectscited by 64
  16. 2021Learning Symbolic Operators for Task and Motion Planningcited by 57
  17. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 57
  18. 2022ToolFlowNet: Robotic Manipulation with Tools via Predicting Tool Flow from Point Cloudscited by 54
  19. 2021SORNet: Spatial Object-Centric Representations for Sequential Manipulationcited by 48
  20. 2022TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulationcited by 47
  21. 2021O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learningcited by 46
  22. 2022Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learningcited by 36
  23. 2021Learning Models as Functionals of Signed-Distance Fields for Manipulation Planningcited by 29
  24. 2021Predicting Stable Configurations for Semantic Placement of Novel Objectscited by 23
  25. 2019Learning Grasp Affordance Reasoning through Semantic Relationscited by 22
  26. 2023Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulationcited by 21
  27. 2021GIFT: Generalizable Interaction-aware Functional Tool Affordances without Labelscited by 19
  28. 2020Transferable Task Execution from Pixels through Deep Planning Domain Learningcited by 17
  29. 2022Planning for Multi-Object Manipulation with Graph Neural Network Relational Classifierscited by 11
  30. 2019CAGE: Context-Aware Grasping Enginecited by 9
  31. 2023You Only Look at One: Category-Level Object Representations for Pose Estimation From a Single Examplecited by 8
  32. 2021Deep Visual Constraints: Neural Implicit Models for Manipulation Planning from Visual Inputcited by 5
  33. 2022Learning Category-Level Manipulation Tasks from Point Clouds with Dynamic Graph CNNscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 9; 2026: 1.

17 papers.

Sort:

By area:

  1. 2024Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstructioncited by 51
  2. 2025Diffusion Models for Robotic Manipulation: A Surveycited by 39
  3. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  4. 2024Keypoint Abstraction using Large Models for Object-Relative Imitation Learningcited by 18
  5. 20243x2: 3D Object Part Segmentation by 2D Semantic Correspondencescited by 10
  6. 2025MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulationcited by 7
  7. 2024One-Shot Manipulation Strategy Learning by Making Contact Analogiescited by 6
  8. 2025Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generationcited by 6
  9. 2024GAPartManip: A Large-scale Part-centric Dataset for Material-Agnostic Articulated Object Manipulationcited by 4
  10. 2025PARTONOMY: Large Multimodal Models with Part-Level Visual Understandingcited by 4
  11. 2025PartInstruct: Part-level Instruction Following for Fine-grained Robot Manipulationcited by 4
  12. 2024Composing Diffusion Policies for Few-shot Learning of Movement Trajectoriescited by 3
5 more
  1. 2025Factorizing Diffusion Policies for Observation Modality Prioritizationcited by 2
  2. 2025Learning to Grasp Anything by Playing with Random Toyscited by 0
  3. 2025Hierarchical Transformers for Unsupervised 3D Shape Abstractioncited by 0
  4. 2025SegGraph: Leveraging Graphs of SAM Segments for Few-Shot 3D Part Segmentationcited by 0
  5. 2026GROW2: Grounding Which and Where for Robot Tool Usecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learning
  2. 2026Skill-Aware Diffusion for Generalizable Robotic Manipulation
  3. 2024Click to Grasp: Zero-Shot Precise Manipulation via Visual Diffusion Descriptors
  4. 2025S2-Diffusion: Generalizing from Instance-level to Category-level Skills in Robot Manipulation
  5. 2026RoTri-Diff: A Spatial Robot-Object Triadic Interaction-Guided Diffusion Model for Bimanual Manipulation
  6. 2025PartDexTOG: Generating Dexterous Task-Oriented Grasping via Language-driven Part Analysis
  7. 2026Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policies
  8. 2024Generative Factor Chaining: Coordinated Manipulation with Diffusion-based Factor Graph