# Reconstruction of Manipulated Garment with Guided Deformation Prior

arXiv: https://arxiv.org/abs/2405.10934 · HTML: https://mlsanity.com/paper/2405.10934/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 63 references that are in the map.
- First posted on arXiv 17 May 2024.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019)
- [DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Model](https://mlsanity.com/paper/2311.09217/) (2023)
- [DiffRF: Rendering-Guided 3D Radiance Field Diffusion](https://mlsanity.com/paper/2212.01206/) (2022)
- [RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation](https://mlsanity.com/paper/2211.09869/) (2022)
- [3D-CODED : 3D Correspondences by Deep Deformation](https://mlsanity.com/paper/1806.05228/) (2018)
- [4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface](https://mlsanity.com/paper/2105.01905/) (2021)
- [BCNet: Learning Body and Cloth Shape from A Single Image](https://mlsanity.com/paper/2004.00214/) (2020)
- [CLOTH3D: Clothed 3D Humans](https://mlsanity.com/paper/1912.02792/) (2019)
- [Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images](https://mlsanity.com/paper/2003.12753/) (2020)
- [NPMs: Neural Parametric Models for 3D Deformable Shapes](https://mlsanity.com/paper/2104.00702/) (2021)
- [Context-aware Human Motion Prediction](https://mlsanity.com/paper/1904.03419/) (2019)
- [PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction](https://mlsanity.com/paper/2302.10668/) (2023)
- [DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data](https://mlsanity.com/paper/1912.04302/) (2019)
- [MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video](https://mlsanity.com/paper/2009.10711/) (2020)
- [Neural Non-Rigid Tracking](https://mlsanity.com/paper/2006.13240/) (2020)
- [GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion](https://mlsanity.com/paper/2104.05177/) (2021)
- [Registering Explicit to Implicit: Towards High-Fidelity Garment mesh Reconstruction from Single Images](https://mlsanity.com/paper/2203.15007/) (2022)
- [ISP: Multi-Layered Garment Draping with Implicit Sewing Patterns](https://mlsanity.com/paper/2305.14100/) (2023)

## Built on by (most cited first, 9 of 9)

- [Single View Garment Reconstruction Using Diffusion Mapping Via Pattern Coordinates](https://mlsanity.com/paper/2504.08353/) (2025)
- [GarmageNet: A Multimodal Generative Framework for Sewing Pattern Design and Generic Garment Modeling](https://mlsanity.com/paper/2504.01483/) (2025)
- [Learning Sewing Patterns via Latent Flow Matching of Implicit Fields](https://mlsanity.com/paper/2601.17740/) (2026)
- [Beyond Static Perception: Integrating Temporal Context into VLMs for Cloth Folding](https://mlsanity.com/paper/2505.07600/) (2025)
- [Spatio-Temporal Garment Reconstruction Using Diffusion Mapping via Pattern Coordinates](https://mlsanity.com/paper/2602.24043/) (2026)
- [GarmentWeaver: Schema-Aware Structured Synthesis for Multimodal Sewing Patterns](https://mlsanity.com/paper/2608.30550/) (2026)
- [DiT-Garment: Garment Dynamics with Diffusion Transformers](https://mlsanity.com/paper/2609.18510/) (2026)
- [SewFusion: Tailored Generation of Topology and Panel-Level Geometry for Sewing Patterns](https://mlsanity.com/paper/2609.23548/) (2026)
- [ClothAtlas: Joint Garment Identification and State Estimation under Self-Occlusion](https://mlsanity.com/paper/2610.16293/) (2026)

## Abstract (arXiv)

Modeling the shape of garments has received much attention, but most existing approaches assume the garments to be worn by someone, which constrains the range of shapes they can assume. In this work, we address shape recovery when garments are being manipulated instead of worn, which gives rise to an even larger range of possible shapes. To this end, we leverage the implicit sewing patterns (ISP) model for garment modeling and extend it by adding a diffusion-based deformation prior to represent these shapes. To recover 3D garment shapes from incomplete 3D point clouds acquired when the garment is folded, we map the points to UV space, in which our priors are learned, to produce partial UV maps, and then fit the priors to recover complete UV maps and 2D to 3D mappings. Experimental results demonstrate the superior reconstruction accuracy of our method compared to previous ones, especially when dealing with large non-rigid deformations arising from the manipulations.