mlsanityData to 7 Oct 2026

General-purpose Clothes Manipulation with Semantic Keypoints

arXiv 2408.08160 ↗First posted 15 Aug 2024cs.RO cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Clothes manipulation is a critical capability for household robots; yet, existing methods are often confined to specific tasks, such as folding or flattening, due to the complex high-dimensional geometry of deformable fabric. This paper presents CLothes mAnipulation with Semantic keyPoints (CLASP) for general-purpose clothes manipulation, which enables the robot to perform diverse manipulation tasks over different types of clothes. The key idea of CLASP is semantic keypoints -- e.g., "right shoulder", "left sleeve", etc. -- a sparse spatial-semantic representation that is salient for both perception and action. Semantic keypoints of clothes can be effectively extracted from depth images and are sufficient to represent a broad range of clothes manipulation policies. CLASP leverages semantic keypoints to bridge LLM-powered task planning and low-level action execution in a two-level hierarchy. Extensive simulation experiments show that CLASP outperforms baseline methods across diverse clothes types in both seen and unseen tasks. Further, experiments with a Kinova dual-arm system on four distinct tasks -- folding, flattening, hanging, and placing -- confirm CLASP's performance on a real robot.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 15 Aug 2024.

Builds on

36 of 45 references.

References · 36

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023Segment Anythingcited by 8,158
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  5. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  6. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  7. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  8. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  9. 2023Scaling Open-Vocabulary Object Detectioncited by 335
  10. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  11. 2018Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluidscited by 219
  12. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 124
24 more
  1. 2020SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulationcited by 118
  2. 2019CLOTH3D: Clothed 3D Humanscited by 114
  3. 2019Unsupervised Learning of Object Keypoints for Perception and Controlcited by 109
  4. 2021FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfoldingcited by 80
  5. 2020Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networkscited by 77
  6. 2020Learning Predictive Representations for Deformable Objects Using Contrastive Estimationcited by 72
  7. 2018Interactive Visual Grounding of Referring Expressions for Human-Robot Interactioncited by 55
  8. 2021FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policycited by 54
  9. 2024AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulationcited by 52
  10. 2022Iterative Residual Policy: for Goal-Conditioned Dynamic Manipulation of Deformable Objectscited by 36
  11. 2022Cloth Funnels: Canonicalized-Alignment for Multi-Purpose Garment Manipulationcited by 28
  12. 2021Skeleton Merger: an Unsupervised Aligned Keypoint Detectorcited by 27
  13. 2020Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulationcited by 25
  14. 2022DaXBench: Benchmarking Deformable Object Manipulation with Differentiable Physicscited by 22
  15. 2023ClothesNet: An Information-Rich 3D Garment Model Repository with Simulated Clothes Environmentcited by 22
  16. 2022Learning Deformable Object Manipulation from Expert Demonstrationscited by 18
  17. 2022DextAIRity: Deformable Manipulation Can be a Breezecited by 17
  18. 2022Offline-Online Learning of Deformation Model for Cable Manipulation with Graph Neural Networkscited by 17
  19. 2022Bag All You Need: Learning a Generalizable Bagging Strategy for Heterogeneous Objectscited by 12
  20. 2024Differentiable Particles for General-Purpose Deformable Object Manipulationcited by 11
  21. 2023Foldsformer: Learning Sequential Multi-Step Cloth Manipulation With Space-Time Attentioncited by 10
  22. 2023A Virtual Reality Framework for Human-Robot Collaboration in Cloth Foldingcited by 7
  23. 20173D Deformable Object Manipulation using Fast Online Gaussian Process Regressioncited by 5
  24. 2022Global Model Learning for Large Deformation Control of Elastic Deformable Linear Objects: An Efficient and Adaptive Approachcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025PinchBot: Long-Horizon Deformable Manipulation with Guided Diffusion Policycited by 1
  2. 2025CLASP: General-Purpose Clothes Manipulation with Semantic Keypointscited by 1
  3. 2026Instant-Fold: In-Context Imitation Learning for Deformable Object Manipulationcited by 1
  4. 2025Deformable Cluster Manipulation via Whole-Arm Policy Learningcited by 0
  5. 2026GROW2: Grounding Which and Where for Robot Tool Usecited by 0