# Robust 3D Garment Digitization from Monocular 2D Images for 3D Virtual Try-On Systems arXiv: https://arxiv.org/abs/2111.15140 ยท HTML: https://mlsanity.com/paper/2111.15140/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Builds on 36 of its 60 references that are in the map. - First posted on arXiv 30 Nov 2021. ## Builds on (most cited first, 30 of 36) - [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019) - [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020) - [VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization](https://mlsanity.com/paper/2103.16874/) (2021) - [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017) - [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020) - [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019) - [Toward Characteristic-Preserving Image-based Virtual Try-On Network](https://mlsanity.com/paper/1807.07688/) (2018) - [Tex2Shape: Detailed Full Human Body Geometry From a Single Image](https://mlsanity.com/paper/1904.08645/) (2019) - [SMPLicit: Topology-aware Generative Model for Clothed People](https://mlsanity.com/paper/2103.06871/) (2021) - [TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style](https://mlsanity.com/paper/2003.04583/) (2020) - [Towards Photo-Realistic Virtual Try-On by Adaptively GeneratingleftrightarrowPreserving Image Content](https://mlsanity.com/paper/2003.05863/) (2020) - [DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images](https://mlsanity.com/paper/1901.07973/) (2019) - [CLOTH3D: Clothed 3D Humans](https://mlsanity.com/paper/1912.02792/) (2019) - [Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction](https://mlsanity.com/paper/2006.08072/) (2020) - [Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images](https://mlsanity.com/paper/2003.12753/) (2020) - [DeepWrinkles: Accurate and Realistic Clothing Modeling](https://mlsanity.com/paper/1808.03417/) (2018) - [Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On](https://mlsanity.com/paper/2105.06462/) (2021) - [Cascaded deep monocular 3D human pose estimation with evolutionary training data](https://mlsanity.com/paper/2006.07778/) (2020) - [Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmark](https://mlsanity.com/paper/1804.01984/) (2018) - [360-Degree Textures of People in Clothing from a Single Image](https://mlsanity.com/paper/1908.07117/) (2019) - [Detailed Human Shape Estimation from a Single Image by Hierarchical Mesh Deformation](https://mlsanity.com/paper/1904.10506/) (2019) - [GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Draping](https://mlsanity.com/paper/1811.10983/) (2018) - [Disentangled Cycle Consistency for Highly-realistic Virtual Try-On](https://mlsanity.com/paper/2103.09479/) (2021) - [Learning to Transfer Texture from Clothing Images to 3D Humans](https://mlsanity.com/paper/2003.02050/) (2020) - [Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing](https://mlsanity.com/paper/2104.07021/) (2021) - [Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness](https://mlsanity.com/paper/2104.13743/) (2021) - [The Conditional Analogy GAN: Swapping Fashion Articles on People Images](https://mlsanity.com/paper/1709.04695/) (2017) - [M2E-Try On Net: Fashion from Model to Everyone](https://mlsanity.com/paper/1811.08599/) (2018) - [SieveNet: A Unified Framework for Robust Image-Based Virtual Try-On](https://mlsanity.com/paper/2001.06265/) (2020) - [Fashion Landmark Detection in the Wild](https://mlsanity.com/paper/1608.03049/) (2016) ## Built on by (most cited first, 13 of 13) - [StableGarment: Garment-Centric Generation via Stable Diffusion](https://mlsanity.com/paper/2403.10783/) (2024) - [GaussianVTON: 3D Human Virtual Try-ON via Multi-Stage Gaussian Splatting Editing with Image Prompting](https://mlsanity.com/paper/2405.07472/) (2024) - [OutfitAnyone: Ultra-high Quality Virtual Try-On for Any Clothing and Any Person](https://mlsanity.com/paper/2407.16224/) (2024) - [OpenFashionCLIP: Vision-and-Language Contrastive Learning with Open-Source Fashion Data](https://mlsanity.com/paper/2309.05551/) (2023) - [Cloth2Tex: A Customized Cloth Texture Generation Pipeline for 3D Virtual Try-On](https://mlsanity.com/paper/2308.04288/) (2023) - [xCloth: Extracting Template-free Textured 3D Clothes from a Monocular Image](https://mlsanity.com/paper/2208.12934/) (2022) - [Leveraging Intrinsic Properties for Non-Rigid Garment Alignment](https://mlsanity.com/paper/2308.09519/) (2023) - [GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing](https://mlsanity.com/paper/2503.08678/) (2025) - [A Generative Multi-Resolution Pyramid and Normal-Conditioning 3D Cloth Draping](https://mlsanity.com/paper/2311.02700/) (2023) - [DeepIron: Predicting Unwarped Garment Texture from a Single Image](https://mlsanity.com/paper/2310.15447/) (2023) - [GC-VTON: Predicting Globally Consistent and Occlusion Aware Local Flows with Neighborhood Integrity Preservation for Virtual Try-on](https://mlsanity.com/paper/2311.04932/) (2023) - [Dress-Me-Up: A Dataset & Method for Self-Supervised 3D Garment Retargeting](https://mlsanity.com/paper/2401.03108/) (2024) - [FabricDiffusion: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Clothing Images](https://mlsanity.com/paper/2410.01801/) (2024) ## Abstract (arXiv) In this paper, we develop a robust 3D garment digitization solution that can generalize well on real-world fashion catalog images with cloth texture occlusions and large body pose variations. We assumed fixed topology parametric template mesh models for known types of garments (e.g., T-shirts, Trousers) and perform mapping of high-quality texture from an input catalog image to UV map panels corresponding to the parametric mesh model of the garment. We achieve this by first predicting a sparse set of 2D landmarks on the boundary of the garments. Subsequently, we use these landmarks to perform Thin-Plate-Spline-based texture transfer on UV map panels. Subsequently, we employ a deep texture inpainting network to fill the large holes (due to view variations & self-occlusions) in TPS output to generate consistent UV maps. Furthermore, to train the supervised deep networks for landmark prediction & texture inpainting tasks, we generated a large set of synthetic data with varying texture and lighting imaged from various views with the human present in a wide variety of poses. Additionally, we manually annotated a small set of fashion catalog images crawled from online fashion e-commerce platforms to finetune. We conduct thorough empirical evaluations and show impressive qualitative results of our proposed 3D garment texture solution on fashion catalog images. Such 3D garment digitization helps us solve the challenging task of enabling 3D Virtual Try-on.