# RaggeDi: Diffusion-based State Estimation of Disordered Rags, Sheets, Towels and Blankets

arXiv: https://arxiv.org/abs/2409.11831 · HTML: https://mlsanity.com/paper/2409.11831/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 18 Sep 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [MeshDiffusion: Score-based Generative 3D Mesh Modeling](https://mlsanity.com/paper/2303.08133/) (2023)
- [SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusion](https://mlsanity.com/paper/2209.03855/) (2022)
- [DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data](https://mlsanity.com/paper/1912.04302/) (2019)
- [Neural Non-Rigid Tracking](https://mlsanity.com/paper/2006.13240/) (2020)
- [GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion](https://mlsanity.com/paper/2104.05177/) (2021)
- [Challenges and Outlook in Robotic Manipulation of Deformable Objects](https://mlsanity.com/paper/2105.01767/) (2021)
- [DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulation](https://mlsanity.com/paper/2312.00583/) (2023)
- [VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation](https://mlsanity.com/paper/2003.09044/) (2020)
- [Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation](https://mlsanity.com/paper/2206.02881/) (2022)
- [Cloth Region Segmentation for Robust Grasp Selection](https://mlsanity.com/paper/2008.05626/) (2020)
- [GarmentTracking: Category-Level Garment Pose Tracking](https://mlsanity.com/paper/2303.13913/) (2023)
- [Tracking Partially-Occluded Deformable Objects while Enforcing Geometric Constraints](https://mlsanity.com/paper/2011.00627/) (2020)
- [Self-supervised Cloth Reconstruction via Action-conditioned Cloth Tracking](https://mlsanity.com/paper/2302.09502/) (2023)
- [Fast Point Cloud to Mesh Reconstruction for Deformable Object Tracking](https://mlsanity.com/paper/2311.02749/) (2023)
- [Bodies Uncovered: Learning to Manipulate Real Blankets Around People via Physics Simulations](https://mlsanity.com/paper/2109.04930/) (2021)
- [Robust Body Exposure (RoBE): A Graph-based Dynamics Modeling Approach to Manipulating Blankets over People](https://mlsanity.com/paper/2304.04822/) (2023)
- [Modal-graph 3D shape servoing of deformable objects with raw point clouds](https://mlsanity.com/paper/2304.08822/) (2023)

## Built on by (most cited first, 1 of 1)

- [Robot Deformable Object Manipulation via NMPC-generated Demonstrations in Deep Reinforcement Learning](https://mlsanity.com/paper/2502.11375/) (2025)

## Abstract (arXiv)

Cloth state estimation is an important problem in robotics. It is essential for the robot to know the accurate state to manipulate cloth and execute tasks such as robotic dressing, stitching, and covering/uncovering human beings. However, estimating cloth state accurately remains challenging due to its high flexibility and self-occlusion. This paper proposes a diffusion model-based pipeline that formulates the cloth state estimation as an image generation problem by representing the cloth state as an RGB image that describes the point-wise translation (translation map) between a pre-defined flattened mesh and the deformed mesh in a canonical space. Then we train a conditional diffusion-based image generation model to predict the translation map based on an observation. Experiments are conducted in both simulation and the real world to validate the performance of our method. Results indicate that our method outperforms two recent methods in both accuracy and speed.