# Single Stage Multi-Pose Virtual Try-On

arXiv: https://arxiv.org/abs/2211.10715 · HTML: https://mlsanity.com/paper/2211.10715/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 49 references that are in the map.
- First posted on arXiv 19 Nov 2022.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [Toward Characteristic-Preserving Image-based Virtual Try-On Network](https://mlsanity.com/paper/1807.07688/) (2018)
- [View Synthesis by Appearance Flow](https://mlsanity.com/paper/1605.03557/) (2016)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [Controllable Person Image Synthesis with Attribute-Decomposed GAN](https://mlsanity.com/paper/2003.12267/) (2020)
- [Towards Photo-Realistic Virtual Try-On by Adaptively GeneratingleftrightarrowPreserving Image Content](https://mlsanity.com/paper/2003.05863/) (2020)
- [Parser-Free Virtual Try-on via Distilling Appearance Flows](https://mlsanity.com/paper/2103.04559/) (2021)
- [Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing](https://mlsanity.com/paper/1703.05446/) (2017)
- [Dense Intrinsic Appearance Flow for Human Pose Transfer](https://mlsanity.com/paper/1903.11326/) (2019)
- [Deep Image Spatial Transformation for Person Image Generation](https://mlsanity.com/paper/2003.00696/) (2020)
- [Towards Multi-pose Guided Virtual Try-on Network](https://mlsanity.com/paper/1902.11026/) (2019)
- [Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing](https://mlsanity.com/paper/2104.14754/) (2021)
- [Do Not Mask What You Do Not Need to Mask: a Parser-Free Virtual Try-On](https://mlsanity.com/paper/2007.02721/) (2020)
- [PISE: Person Image Synthesis and Editing with Decoupled GAN](https://mlsanity.com/paper/2103.04023/) (2021)

## Built on by (most cited first, 1 of 1)

- [FashionFlow: Leveraging Diffusion Models for Dynamic Fashion Video Synthesis from Static Imagery](https://mlsanity.com/paper/2310.00106/) (2023)

## Abstract (arXiv)

Multi-pose virtual try-on (MPVTON) aims to fit a target garment onto a person at a target pose. Compared to traditional virtual try-on (VTON) that fits the garment but keeps the pose unchanged, MPVTON provides a better try-on experience, but is also more challenging due to the dual garment and pose editing objectives. Existing MPVTON methods adopt a pipeline comprising three disjoint modules including a target semantic layout prediction module, a coarse try-on image generator and a refinement try-on image generator. These models are trained separately, leading to sub-optimal model training and unsatisfactory results. In this paper, we propose a novel single stage model for MPVTON. Key to our model is a parallel flow estimation module that predicts the flow fields for both person and garment images conditioned on the target pose. The predicted flows are subsequently used to warp the appearance feature maps of the person and the garment images to construct a style map. The map is then used to modulate the target pose's feature map for target try-on image generation. With the parallel flow estimation design, our model can be trained end-to-end in a single stage and is more computationally efficient, resulting in new SOTA performance on existing MPVTON benchmarks. We further introduce multi-task training and demonstrate that our model can also be applied for traditional VTON and pose transfer tasks and achieve comparable performance to SOTA specialized models on both tasks.