# Improving Diffusion Models for Authentic Virtual Try-on in the Wild

arXiv: https://arxiv.org/abs/2403.05139 · HTML: https://mlsanity.com/paper/2403.05139/

## Key facts

- Cited by 109 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 56 of its 60 references that are in the map.
- First posted on arXiv 8 Mar 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 56)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)

## Built on by (most cited first, 30 of 109)

- [IMAGDressing-v1: Customizable Virtual Dressing](https://mlsanity.com/paper/2407.12705/) (2024)
- [DreamO: A Unified Framework for Image Customization](https://mlsanity.com/paper/2504.16915/) (2025)
- [Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks](https://mlsanity.com/paper/2501.15891/) (2025)
- [BooW-VTON: Boosting In-the-Wild Virtual Try-On via Mask-Free Pseudo Data Training](https://mlsanity.com/paper/2408.06047/) (2024)
- [Learning Flow Fields in Attention for Controllable Person Image Generation](https://mlsanity.com/paper/2412.08486/) (2024)
- [AnyFit: Controllable Virtual Try-on for Any Combination of Attire Across Any Scenario](https://mlsanity.com/paper/2405.18172/) (2024)
- [OutfitAnyone: Ultra-high Quality Virtual Try-On for Any Clothing and Any Person](https://mlsanity.com/paper/2407.16224/) (2024)
- [MagicTryOn: Harnessing Diffusion Transformer for Garment-Preserving Video Virtual Try-on](https://mlsanity.com/paper/2505.21325/) (2025)
- [RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Models](https://mlsanity.com/paper/2506.18369/) (2025)
- [Pursuing Temporal-Consistent Video Virtual Try-On via Dynamic Pose Interaction](https://mlsanity.com/paper/2505.16980/) (2025)
- [MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing](https://mlsanity.com/paper/2502.21291/) (2025)
- [OmniVTON: Training-Free Universal Virtual Try-On](https://mlsanity.com/paper/2507.15037/) (2025)
- [MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generation](https://mlsanity.com/paper/2511.22989/) (2025)
- [Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On](https://mlsanity.com/paper/2505.16977/) (2025)
- [Voost: A Unified and Scalable Diffusion Transformer for Bidirectional Virtual Try-On and Try-Off](https://mlsanity.com/paper/2508.04825/) (2025)
- [JCo-MVTON: Jointly Controllable Multi-Modal Diffusion Transformer for Mask-Free Virtual Try-on](https://mlsanity.com/paper/2508.17614/) (2025)
- [Eevee: Towards Close-up High-resolution Video-based Virtual Try-on](https://mlsanity.com/paper/2511.18957/) (2025)
- [MC-VTON: Minimal Control Virtual Try-On Diffusion Transformer](https://mlsanity.com/paper/2501.03630/) (2025)
- [Shining Yourself: High-Fidelity Ornaments Virtual Try-on with Diffusion Model](https://mlsanity.com/paper/2503.16065/) (2025)
- [RegionE: Adaptive Region-Aware Generation for Efficient Image Editing](https://mlsanity.com/paper/2510.25590/) (2025)
- [Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing](https://mlsanity.com/paper/2403.14828/) (2024)
- [Controllable Human Image Generation with Personalized Multi-Garments](https://mlsanity.com/paper/2411.16801/) (2024)
- [Try Harder: Hard Sample Generation and Learning for Clothes-Changing Person Re-ID](https://mlsanity.com/paper/2507.11119/) (2025)
- [Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generation](https://mlsanity.com/paper/2609.11638/) (2026)
- [EmojiDiff: Advanced Facial Expression Control with High Identity Preservation in Portrait Generation](https://mlsanity.com/paper/2412.01254/) (2024)
- [ITVTON: Virtual Try-On Diffusion Transformer Based on Integrated Image and Text](https://mlsanity.com/paper/2501.16757/) (2025)
- [Inverse Virtual Try-On: Generating Multi-Category Product-Style Images from Clothed Individuals](https://mlsanity.com/paper/2505.21062/) (2025)
- [Loom: Diffusion-Transformer for Interleaved Generation](https://mlsanity.com/paper/2512.18254/) (2025)
- [iTryOn: Mastering Interactive Video Virtual Try-On with Spatial-Semantic Guidance](https://mlsanity.com/paper/2605.21431/) (2026)
- [Creating Your Editable 3D Photorealistic Avatar with Tetrahedron-constrained Gaussian Splatting](https://mlsanity.com/paper/2504.20403/) (2025)

## Abstract (arXiv)

This paper considers image-based virtual try-on, which renders an image of a person wearing a curated garment, given a pair of images depicting the person and the garment, respectively. Previous works adapt existing exemplar-based inpainting diffusion models for virtual try-on to improve the naturalness of the generated visuals compared to other methods (e.g., GAN-based), but they fail to preserve the identity of the garments. To overcome this limitation, we propose a novel diffusion model that improves garment fidelity and generates authentic virtual try-on images. Our method, coined IDM-VTON, uses two different modules to encode the semantics of garment image; given the base UNet of the diffusion model, 1) the high-level semantics extracted from a visual encoder are fused to the cross-attention layer, and then 2) the low-level features extracted from parallel UNet are fused to the self-attention layer. In addition, we provide detailed textual prompts for both garment and person images to enhance the authenticity of the generated visuals. Finally, we present a customization method using a pair of person-garment images, which significantly improves fidelity and authenticity. Our experimental results show that our method outperforms previous approaches (both diffusion-based and GAN-based) in preserving garment details and generating authentic virtual try-on images, both qualitatively and quantitatively. Furthermore, the proposed customization method demonstrates its effectiveness in a real-world scenario. More visualizations are available in our project page: https://idm-vton.github.io