# Per Garment Capture and Synthesis for Real-time Virtual Try-on

arXiv: https://arxiv.org/abs/2109.04654 · HTML: https://mlsanity.com/paper/2109.04654/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 56 references that are in the map.
- First posted on arXiv 10 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017)
- [Toward Characteristic-Preserving Image-based Virtual Try-On Network](https://mlsanity.com/paper/1807.07688/) (2018)
- [Towards Photo-Realistic Virtual Try-On by Adaptively GeneratingleftrightarrowPreserving Image Content](https://mlsanity.com/paper/2003.05863/) (2020)
- [Instance-level Human Parsing via Part Grouping Network](https://mlsanity.com/paper/1808.00157/) (2018)
- [DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images](https://mlsanity.com/paper/1901.07973/) (2019)
- [Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images](https://mlsanity.com/paper/2003.12753/) (2020)
- [Towards Multi-pose Guided Virtual Try-on Network](https://mlsanity.com/paper/1902.11026/) (2019)
- [DeepWrinkles: Accurate and Realistic Clothing Modeling](https://mlsanity.com/paper/1808.03417/) (2018)
- [Detailed, accurate, human shape estimation from clothed 3D scan sequences](https://mlsanity.com/paper/1703.04454/) (2017)
- [Learning-Based Animation of Clothing for Virtual Try-On](https://mlsanity.com/paper/1903.07190/) (2019)
- [SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing](https://mlsanity.com/paper/2007.11610/) (2020)
- [Graphonomy: Universal Human Parsing via Graph Transfer Learning](https://mlsanity.com/paper/1904.04536/) (2019)
- [Devil in the Details: Towards Accurate Single and Multiple Human Parsing](https://mlsanity.com/paper/1809.05996/) (2018)
- [Learning to Transfer Texture from Clothing Images to 3D Humans](https://mlsanity.com/paper/2003.02050/) (2020)
- [A Generative Model of People in Clothing](https://mlsanity.com/paper/1705.04098/) (2017)
- [The Conditional Analogy GAN: Swapping Fashion Articles on People Images](https://mlsanity.com/paper/1709.04695/) (2017)
- [ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on](https://mlsanity.com/paper/2012.10495/) (2020)
- [3D Virtual Garment Modeling from RGB Images](https://mlsanity.com/paper/1908.00114/) (2019)
- [Down to the Last Detail: Virtual Try-on with Detail Carving](https://mlsanity.com/paper/1912.06324/) (2019)

## Built on by (most cited first, 5 of 5)

- [Leveraging Intrinsic Properties for Non-Rigid Garment Alignment](https://mlsanity.com/paper/2308.09519/) (2023)
- [SiCo: An Interactive Size-Controllable Virtual Try-On Approach for Informed Decision-Making](https://mlsanity.com/paper/2408.02803/) (2024)
- [Low-Barrier Dataset Collection with Real Human Body for Interactive Per-Garment Virtual Try-On](https://mlsanity.com/paper/2506.10468/) (2025)
- [Real-Time Per-Garment Virtual Try-On with Temporal Consistency for Loose-Fitting Garments](https://mlsanity.com/paper/2506.12348/) (2025)
- [Diffusion Model-Based Size Variable Virtual Try-On Technology and Evaluation Method](https://mlsanity.com/paper/2504.00562/) (2025)

## Abstract (arXiv)

Virtual try-on is a promising application of computer graphics and human computer interaction that can have a profound real-world impact especially during this pandemic. Existing image-based works try to synthesize a try-on image from a single image of a target garment, but it inherently limits the ability to react to possible interactions. It is difficult to reproduce the change of wrinkles caused by pose and body size change, as well as pulling and stretching of the garment by hand. In this paper, we propose an alternative per garment capture and synthesis workflow to handle such rich interactions by training the model with many systematically captured images. Our workflow is composed of two parts: garment capturing and clothed person image synthesis. We designed an actuated mannequin and an efficient capturing process that collects the detailed deformations of the target garments under diverse body sizes and poses. Furthermore, we proposed to use a custom-designed measurement garment, and we captured paired images of the measurement garment and the target garments. We then learn a mapping between the measurement garment and the target garments using deep image-to-image translation. The customer can then try on the target garments interactively during online shopping.