# ITVTON: Virtual Try-On Diffusion Transformer Based on Integrated Image and Text

arXiv: https://arxiv.org/abs/2501.16757 · HTML: https://mlsanity.com/paper/2501.16757/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization](https://mlsanity.com/paper/2103.16874/) (2021)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [Self-Correction for Human Parsing](https://mlsanity.com/paper/1910.09777/) (2019)
- [StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On](https://mlsanity.com/paper/2312.01725/) (2023)
- [In-Context LoRA for Diffusion Transformers](https://mlsanity.com/paper/2410.23775/) (2024)
- [LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-On](https://mlsanity.com/paper/2305.13501/) (2023)
- [OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on](https://mlsanity.com/paper/2403.01779/) (2024)
- [TryOnDiffusion: A Tale of Two UNets](https://mlsanity.com/paper/2306.08276/) (2023)
- [Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flow](https://mlsanity.com/paper/2308.06101/) (2023)
- [GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learning](https://mlsanity.com/paper/2303.13756/) (2023)
- [Improving Diffusion Models for Authentic Virtual Try-on in the Wild](https://mlsanity.com/paper/2403.05139/) (2024)
- [Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models](https://mlsanity.com/paper/2310.06313/) (2023)
- [CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models](https://mlsanity.com/paper/2407.15886/) (2024)
- [IMAGDressing-v1: Customizable Virtual Dressing](https://mlsanity.com/paper/2407.12705/) (2024)
- [Towards Multi-pose Guided Virtual Try-on Network](https://mlsanity.com/paper/1902.11026/) (2019)
- [Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models](https://mlsanity.com/paper/2407.02482/) (2024)

## Built on by (most cited first, 4 of 4)

- [EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer](https://mlsanity.com/paper/2503.07027/) (2025)
- [VTBench: Comprehensive Benchmark Suite Towards Real-World Virtual Try-on Models](https://mlsanity.com/paper/2505.19571/) (2025)
- [The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignment](https://mlsanity.com/paper/2511.20614/) (2025)
- [PROMO: Promptable Outfitting for Efficient High-Fidelity Virtual Try-On](https://mlsanity.com/paper/2603.11675/) (2026)

## Abstract (arXiv)

Virtual try-on, which aims to seamlessly fit garments onto person images, has recently seen significant progress with diffusion-based models. However, existing methods commonly resort to duplicated backbones or additional image encoders to extract garment features, which increases computational overhead and network complexity. In this paper, we propose ITVTON, an efficient framework that leverages the Diffusion Transformer (DiT) as its single generator to improve image fidelity. By concatenating garment and person images along the width dimension and incorporating textual descriptions from both, ITVTON effectively captures garment-person interactions while preserving realism. To further reduce computational cost, we restrict training to the attention parameters within a single Diffusion Transformer (Single-DiT) block. Extensive experiments demonstrate that ITVTON surpasses baseline methods both qualitatively and quantitatively, setting a new standard for virtual try-on. Moreover, experiments on 10,257 image pairs from IGPair confirm its robustness in real-world scenarios.