# GraphGarment: Learning Garment Dynamics for Bimanual Cloth Manipulation Tasks

arXiv: https://arxiv.org/abs/2503.05817 · HTML: https://mlsanity.com/paper/2503.05817/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 4 Mar 2025.

## Builds on (most cited first, 20 of 20)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Sim-to-Real Reinforcement Learning for Deformable Object Manipulation](https://mlsanity.com/paper/1806.07851/) (2018)
- [GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion](https://mlsanity.com/paper/2104.05177/) (2021)
- [Challenges and Outlook in Robotic Manipulation of Deformable Objects](https://mlsanity.com/paper/2105.01767/) (2021)
- [UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondence](https://mlsanity.com/paper/2405.06903/) (2024)
- [GarmentTracking: Category-Level Garment Pose Tracking](https://mlsanity.com/paper/2303.13913/) (2023)
- [A Grasping-centered Analysis for Cloth Manipulation](https://mlsanity.com/paper/1906.08202/) (2019)
- [Learning to Grasp Clothing Structural Regions for Garment Manipulation Tasks](https://mlsanity.com/paper/2306.14553/) (2023)
- [Foldsformer: Learning Sequential Multi-Step Cloth Manipulation With Space-Time Attention](https://mlsanity.com/paper/2301.03003/) (2023)
- [TraKDis: A Transformer-based Knowledge Distillation Approach for Visual Reinforcement Learning with Application to Cloth Manipulation](https://mlsanity.com/paper/2401.13362/) (2024)
- [Manipulation of Granular Materials by Learning Particle Interactions](https://mlsanity.com/paper/2111.02274/) (2021)
- [Do You Need a Hand? -- a Bimanual Robotic Dressing Assistance Scheme](https://mlsanity.com/paper/2301.02749/) (2023)
- [A Backbone for Long-Horizon Robot Task Understanding](https://mlsanity.com/paper/2408.01334/) (2024)
- [Bimanual Deformable Bag Manipulation Using a Structure-of-Interest Based Neural Dynamics Model](https://mlsanity.com/paper/2401.11432/) (2024)
- [G.O.G: A Versatile Gripper-On-Gripper Design for Bimanual Cloth Manipulation with a Single Robotic Arm](https://mlsanity.com/paper/2401.10702/) (2024)

## Built on by (most cited first, 3 of 3)

- [Reactive In-Air Clothing Manipulation with Confidence-Aware Dense Correspondence and Visuotactile Affordance](https://mlsanity.com/paper/2509.03889/) (2025)
- [TRACER: Texture-Robust Affordance Chain-of-Thought for Deformable-Object Refinement](https://mlsanity.com/paper/2601.20208/) (2026)
- [Phase-Conditioned Imitation Learning with Autonomous Failure Recovery for Robust Deformable Object Manipulation](https://mlsanity.com/paper/2605.29407/) (2026)

## Abstract (arXiv)

Physical manipulation of garments is often crucial when performing fabric-related tasks, such as hanging garments. However, due to the deformable nature of fabrics, these operations remain a significant challenge for robots in household, healthcare, and industrial environments. In this paper, we propose GraphGarment, a novel approach that models garment dynamics based on robot control inputs and applies the learned dynamics model to facilitate garment manipulation tasks such as hanging. Specifically, we use graphs to represent the interactions between the robot end-effector and the garment. GraphGarment uses a graph neural network (GNN) to learn a dynamics model that can predict the next garment state given the current state and input action in simulation. To address the substantial sim-to-real gap, we propose a residual model that compensates for garment state prediction errors, thereby improving real-world performance. The garment dynamics model is then applied to a model-based action sampling strategy, where it is utilized to manipulate the garment to a reference pre-hanging configuration for garment-hanging tasks. We conducted four experiments using six types of garments to validate our approach in both simulation and real-world settings. In simulation experiments, GraphGarment achieves better garment state prediction performance, with a prediction error 0.46 cm lower than the best baseline. Our approach also demonstrates improved performance in the garment-hanging simulation experiment with enhancements of 12%, 24%, and 10%, respectively. Moreover, real-world robot experiments confirm the robustness of sim-to-real transfer, with an error increase of 0.17 cm compared to simulation results. Supplementary material is available at:https://sites.google.com/view/graphgarment.