# Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On

arXiv: https://arxiv.org/abs/2105.06462 · HTML: https://mlsanity.com/paper/2105.06462/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 76 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 30 of 34)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017)
- [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020)
- [Learning Shape Templates with Structured Implicit Functions](https://mlsanity.com/paper/1904.06447/) (2019)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Learning to Reconstruct People in Clothing from a Single RGB Camera](https://mlsanity.com/paper/1903.05885/) (2019)
- [DeepCap: Monocular Human Performance Capture Using Weak Supervision](https://mlsanity.com/paper/2003.08325/) (2020)
- [CLOTH3D: Clothed 3D Humans](https://mlsanity.com/paper/1912.02792/) (2019)
- [Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images](https://mlsanity.com/paper/2003.12753/) (2020)
- [DeepWrinkles: Accurate and Realistic Clothing Modeling](https://mlsanity.com/paper/1808.03417/) (2018)
- [Detailed, accurate, human shape estimation from clothed 3D scan sequences](https://mlsanity.com/paper/1703.04454/) (2017)
- [Learning-Based Animation of Clothing for Virtual Try-On](https://mlsanity.com/paper/1903.07190/) (2019)
- [SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing](https://mlsanity.com/paper/2007.11610/) (2020)
- [GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Draping](https://mlsanity.com/paper/1811.10983/) (2018)
- [Moulding Humans: Non-parametric 3D Human Shape Estimation from Single Images](https://mlsanity.com/paper/1908.00439/) (2019)
- [3DPeople: Modeling the Geometry of Dressed Humans](https://mlsanity.com/paper/1904.04571/) (2019)
- [Learning a Shared Shape Space for Multimodal Garment Design](https://mlsanity.com/paper/1806.11335/) (2018)
- [Fully Convolutional Graph Neural Networks for Parametric Virtual Try-On](https://mlsanity.com/paper/2009.04592/) (2020)
- [SimulCap : Single-View Human Performance Capture with Cloth Simulation](https://mlsanity.com/paper/1903.06323/) (2019)
- [A Generative Model of People in Clothing](https://mlsanity.com/paper/1705.04098/) (2017)
- [A Pixel-Based Framework for Data-Driven Clothing](https://mlsanity.com/paper/1812.01677/) (2018)

## Built on by (most cited first, 30 of 81)

- [Dress Code: High-Resolution Multi-Category Virtual Try-On](https://mlsanity.com/paper/2204.08532/) (2022)
- [ARAH: Animatable Volume Rendering of Articulated Human SDFs](https://mlsanity.com/paper/2210.10036/) (2022)
- [TADA! Text to Animatable Digital Avatars](https://mlsanity.com/paper/2308.10899/) (2023)
- [HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting](https://mlsanity.com/paper/2311.17061/) (2023)
- [HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamics](https://mlsanity.com/paper/2212.07242/) (2022)
- [Modeling Clothing as a Separate Layer for an Animatable Human Avatar](https://mlsanity.com/paper/2106.14879/) (2021)
- [COAP: Compositional Articulated Occupancy of People](https://mlsanity.com/paper/2204.06184/) (2022)
- [PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence](https://mlsanity.com/paper/2203.01754/) (2022)
- [DrapeNet: Garment Generation and Self-Supervised Draping](https://mlsanity.com/paper/2211.11277/) (2022)
- [DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation](https://mlsanity.com/paper/2311.12194/) (2023)
- [Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks](https://mlsanity.com/paper/2205.01355/) (2022)
- [Neural Cloth Simulation](https://mlsanity.com/paper/2212.11220/) (2022)
- [ISP: Multi-Layered Garment Draping with Implicit Sewing Patterns](https://mlsanity.com/paper/2305.14100/) (2023)
- [SPAMs: Structured Implicit Parametric Models](https://mlsanity.com/paper/2201.08141/) (2022)
- [DIG: Draping Implicit Garment over the Human Body](https://mlsanity.com/paper/2209.10845/) (2022)
- [Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video](https://mlsanity.com/paper/2409.08189/) (2024)
- [Garment Recovery with Shape and Deformation Priors](https://mlsanity.com/paper/2311.10356/) (2023)
- [ContourCraft: Learning to Resolve Intersections in Neural Multi-Garment Simulations](https://mlsanity.com/paper/2405.09522/) (2024)
- [GaussianVTON: 3D Human Virtual Try-ON via Multi-Stage Gaussian Splatting Editing with Image Prompting](https://mlsanity.com/paper/2405.07472/) (2024)
- [CloSET: Modeling Clothed Humans on Continuous Surface with Explicit Template Decomposition](https://mlsanity.com/paper/2304.03167/) (2023)
- [Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprint](https://mlsanity.com/paper/2410.15391/) (2024)
- [Dressing Avatars: Deep Photorealistic Appearance for Physically Simulated Clothing](https://mlsanity.com/paper/2206.15470/) (2022)
- [Motion Guided Deep Dynamic 3D Garments](https://mlsanity.com/paper/2209.11449/) (2022)
- [UV Gaussians: Joint Learning of Mesh Deformation and Gaussian Textures for Human Avatar Modeling](https://mlsanity.com/paper/2403.11589/) (2024)
- [Towards Multi-Layered 3D Garments Animation](https://mlsanity.com/paper/2305.10418/) (2023)
- [N-Cloth: Predicting 3D Cloth Deformation with Mesh-Based Networks](https://mlsanity.com/paper/2112.06397/) (2021)
- [Robust 3D Garment Digitization from Monocular 2D Images for 3D Virtual Try-On Systems](https://mlsanity.com/paper/2111.15140/) (2021)
- [BodyMap: Learning Full-Body Dense Correspondence Map](https://mlsanity.com/paper/2205.09111/) (2022)
- [NeuralTailor: Reconstructing Sewing Pattern Structures from 3D Point Clouds of Garments](https://mlsanity.com/paper/2201.13063/) (2022)
- [VTON 360: High-Fidelity Virtual Try-On from Any Viewing Direction](https://mlsanity.com/paper/2503.12165/) (2025)

## Abstract (arXiv)

We propose a new generative model for 3D garment deformations that enables us to learn, for the first time, a data-driven method for virtual try-on that effectively addresses garment-body collisions. In contrast to existing methods that require an undesirable postprocessing step to fix garment-body interpenetrations at test time, our approach directly outputs 3D garment configurations that do not collide with the underlying body. Key to our success is a new canonical space for garments that removes pose-and-shape deformations already captured by a new diffused human body model, which extrapolates body surface properties such as skinning weights and blendshapes to any 3D point. We leverage this representation to train a generative model with a novel self-supervised collision term that learns to reliably solve garment-body interpenetrations. We extensively evaluate and compare our results with recently proposed data-driven methods, and show that our method is the first to successfully address garment-body contact in unseen body shapes and motions, without compromising realism and detail.