# SPG-VTON: Semantic Prediction Guidance for Multi-pose Virtual Try-on

arXiv: https://arxiv.org/abs/2108.01578 · HTML: https://mlsanity.com/paper/2108.01578/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 58 references that are in the map.
- First posted on arXiv 3 Aug 2021.

## Builds on (most cited first, 30 of 41)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017)
- [Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro](https://mlsanity.com/paper/1701.07717/) (2017)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Joint Discriminative and Generative Learning for Person Re-identification](https://mlsanity.com/paper/1904.07223/) (2019)
- [Toward Characteristic-Preserving Image-based Virtual Try-On Network](https://mlsanity.com/paper/1807.07688/) (2018)
- [Self-Correction for Human Parsing](https://mlsanity.com/paper/1910.09777/) (2019)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [Towards Photo-Realistic Virtual Try-On by Adaptively GeneratingleftrightarrowPreserving Image Content](https://mlsanity.com/paper/2003.05863/) (2020)
- [Dual-Path Convolutional Image-Text Embeddings with Instance Loss](https://mlsanity.com/paper/1711.05535/) (2017)
- [Parser-Free Virtual Try-on via Distilling Appearance Flows](https://mlsanity.com/paper/2103.04559/) (2021)
- [Convolutional neural network architecture for geometric matching](https://mlsanity.com/paper/1703.05593/) (2017)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing](https://mlsanity.com/paper/1904.09709/) (2019)
- [Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing](https://mlsanity.com/paper/1703.05446/) (2017)
- [Disentangled Person Image Generation](https://mlsanity.com/paper/1712.02621/) (2017)
- [Towards Multi-pose Guided Virtual Try-on Network](https://mlsanity.com/paper/1902.11026/) (2019)

## Built on by (most cited first, 12 of 12)

- [SGDiff: A Style Guided Diffusion Model for Fashion Synthesis](https://mlsanity.com/paper/2308.07605/) (2023)
- [Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video Generation](https://mlsanity.com/paper/2207.03714/) (2022)
- [OccluMix: Towards De-Occlusion Virtual Try-on by Semantically-Guided Mixup](https://mlsanity.com/paper/2301.00965/) (2023)
- [A Two-stage Personalized Virtual Try-on Framework with Shape Control and Texture Guidance](https://mlsanity.com/paper/2312.15480/) (2023)
- [PG-VTON: A Novel Image-Based Virtual Try-On Method via Progressive Inference Paradigm](https://mlsanity.com/paper/2304.08956/) (2023)
- [Limb-Aware Virtual Try-On Network with Progressive Clothing Warping](https://mlsanity.com/paper/2503.14074/) (2025)
- [Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing](https://mlsanity.com/paper/2111.12346/) (2021)
- [PFDM: Parser-Free Virtual Try-on via Diffusion Model](https://mlsanity.com/paper/2402.03047/) (2024)
- [DI-Net : Decomposed Implicit Garment Transfer Network for Digital Clothed 3D Human](https://mlsanity.com/paper/2311.16818/) (2023)
- [Dual Stage Stylization Modulation for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2304.09347/) (2023)
- [DiffusionTrend: A Minimalist Approach to Virtual Fashion Try-On](https://mlsanity.com/paper/2412.14465/) (2024)
- [Neural Clothing Tryer: Customized Virtual Try-On via Semantic Enhancement and Controlling Diffusion Model](https://mlsanity.com/paper/2601.22838/) (2026)

## Abstract (arXiv)

Image-based virtual try-on is challenging in fitting a target in-shop clothes into a reference person under diverse human poses. Previous works focus on preserving clothing details ( e.g., texture, logos, patterns ) when transferring desired clothes onto a target person under a fixed pose. However, the performances of existing methods significantly dropped when extending existing methods to multi-pose virtual try-on. In this paper, we propose an end-to-end Semantic Prediction Guidance multi-pose Virtual Try-On Network (SPG-VTON), which could fit the desired clothing into a reference person under arbitrary poses. Concretely, SPG-VTON is composed of three sub-modules. First, a Semantic Prediction Module (SPM) generates the desired semantic map. The predicted semantic map provides more abundant guidance to locate the desired clothes region and produce a coarse try-on image. Second, a Clothes Warping Module (CWM) warps in-shop clothes to the desired shape according to the predicted semantic map and the desired pose. Specifically, we introduce a conductible cycle consistency loss to alleviate the misalignment in the clothes warping process. Third, a Try-on Synthesis Module (TSM) combines the coarse result and the warped clothes to generate the final virtual try-on image, preserving details of the desired clothes and under the desired pose. Besides, we introduce a face identity loss to refine the facial appearance and maintain the identity of the final virtual try-on result at the same time. We evaluate the proposed method on the most massive multi-pose dataset (MPV) and the DeepFashion dataset. The qualitative and quantitative experiments show that SPG-VTON is superior to the state-of-the-art methods and is robust to the data noise, including background and accessory changes, i.e., hats and handbags, showing good scalability to the real-world scenario.