# Learning to Transfer Texture from Clothing Images to 3D Humans

arXiv: https://arxiv.org/abs/2003.02050 · HTML: https://mlsanity.com/paper/2003.02050/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2020.

## Built on by (most cited first, 30 of 58)

- [Dress Code: High-Resolution Multi-Category Virtual Try-On](https://mlsanity.com/paper/2204.08532/) (2022)
- [Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors](https://mlsanity.com/paper/2103.17265/) (2021)
- [Style-Based Global Appearance Flow for Virtual Try-On](https://mlsanity.com/paper/2204.01046/) (2022)
- [Single Stage Virtual Try-on via Deformable Attention Flows](https://mlsanity.com/paper/2207.09161/) (2022)
- [StylePeople: A Generative Model of Fullbody Human Avatars](https://mlsanity.com/paper/2104.08363/) (2021)
- [Body Meshes as Points](https://mlsanity.com/paper/2105.02467/) (2021)
- [ClipMatrix: Text-controlled Creation of 3D Textured Meshes](https://mlsanity.com/paper/2109.12922/) (2021)
- [Point-Based Modeling of Human Clothing](https://mlsanity.com/paper/2104.08230/) (2021)
- [Learned Vertex Descent: A New Direction for 3D Human Model Fitting](https://mlsanity.com/paper/2205.06254/) (2022)
- [3D Human Texture Estimation from a Single Image with Transformers](https://mlsanity.com/paper/2109.02563/) (2021)
- [M3D-VTON: A Monocular-to-3D Virtual Try-On Network](https://mlsanity.com/paper/2108.05126/) (2021)
- [SMPLitex: A Generative Model and Dataset for 3D Human Texture Estimation from Single Image](https://mlsanity.com/paper/2309.01855/) (2023)
- [ClothFormer:Taming Video Virtual Try-on in All Module](https://mlsanity.com/paper/2204.12151/) (2022)
- [GaussianVTON: 3D Human Virtual Try-ON via Multi-Stage Gaussian Splatting Editing with Image Prompting](https://mlsanity.com/paper/2405.07472/) (2024)
- [Semi-supervised Synthesis of High-Resolution Editable Textures for 3D Humans](https://mlsanity.com/paper/2103.17266/) (2021)
- [Structured 3D Features for Reconstructing Controllable Avatars](https://mlsanity.com/paper/2212.06820/) (2022)
- [Cloning Outfits from Real-World Images to 3D Characters for Generalizable Person Re-Identification](https://mlsanity.com/paper/2204.02611/) (2022)
- [Detailed Avatar Recovery from Single Image](https://mlsanity.com/paper/2108.02931/) (2021)
- [OutfitAnyone: Ultra-high Quality Virtual Try-On for Any Clothing and Any Person](https://mlsanity.com/paper/2407.16224/) (2024)
- [Fine Detailed Texture Learning for 3D Meshes with Generative Models](https://mlsanity.com/paper/2203.09362/) (2022)
- [Robust 3D Garment Digitization from Monocular 2D Images for 3D Virtual Try-On Systems](https://mlsanity.com/paper/2111.15140/) (2021)
- [Flow Guided Transformable Bottleneck Networks for Motion Retargeting](https://mlsanity.com/paper/2106.07771/) (2021)
- [Multi-Person 3D Pose and Shape Estimation via Inverse Kinematics and Refinement](https://mlsanity.com/paper/2210.13529/) (2022)
- [Human Image Generation: A Comprehensive Survey](https://mlsanity.com/paper/2212.08896/) (2022)
- [VTON 360: High-Fidelity Virtual Try-On from Any Viewing Direction](https://mlsanity.com/paper/2503.12165/) (2025)
- [SMPLer: Taming Transformers for Monocular 3D Human Shape and Pose Estimation](https://mlsanity.com/paper/2404.15276/) (2024)
- [A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos](https://mlsanity.com/paper/2103.15596/) (2021)
- [Generating Texture for 3D Human Avatar from a Single Image using Sampling and Refinement Networks](https://mlsanity.com/paper/2305.00936/) (2023)
- [DreamVTON: Customizing 3D Virtual Try-on with Personalized Diffusion Models](https://mlsanity.com/paper/2407.16511/) (2024)
- [Cloth2Tex: A Customized Cloth Texture Generation Pipeline for 3D Virtual Try-On](https://mlsanity.com/paper/2308.04288/) (2023)

## Abstract (arXiv)

In this paper, we present a simple yet effective method to automatically transfer textures of clothing images (front and back) to 3D garments worn on top SMPL, in real time. We first automatically compute training pairs of images with aligned 3D garments using a custom non-rigid 3D to 2D registration method, which is accurate but slow. Using these pairs, we learn a mapping from pixels to the 3D garment surface. Our idea is to learn dense correspondences from garment image silhouettes to a 2D-UV map of a 3D garment surface using shape information alone, completely ignoring texture, which allows us to generalize to the wide range of web images. Several experiments demonstrate that our model is more accurate than widely used baselines such as thin-plate-spline warping and image-to-image translation networks while being orders of magnitude faster. Our model opens the door for applications such as virtual try-on, and allows for generation of 3D humans with varied textures which is necessary for learning.