# TryOnDiffusion: A Tale of Two UNets

arXiv: https://arxiv.org/abs/2306.08276 · HTML: https://mlsanity.com/paper/2306.08276/

## Key facts

- Cited by 141 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 56 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Novel View Synthesis with Diffusion Models](https://mlsanity.com/paper/2210.04628/) (2022)
- [VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization](https://mlsanity.com/paper/2103.16874/) (2021)
- [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017)
- [Toward Characteristic-Preserving Image-based Virtual Try-On Network](https://mlsanity.com/paper/1807.07688/) (2018)
- [FILM: Frame Interpolation for Large Motion](https://mlsanity.com/paper/2202.04901/) (2022)
- [Controllable Person Image Synthesis with Attribute-Decomposed GAN](https://mlsanity.com/paper/2003.12267/) (2020)

## Built on by (most cited first, 30 of 141)

- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions](https://mlsanity.com/paper/2402.17485/) (2024)
- [Diffusion Model-Based Image Editing: A Survey](https://mlsanity.com/paper/2402.17525/) (2024)
- [StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On](https://mlsanity.com/paper/2312.01725/) (2023)
- [OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on](https://mlsanity.com/paper/2403.01779/) (2024)
- [Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances](https://mlsanity.com/paper/2410.18775/) (2024)
- [Improving Diffusion Models for Authentic Virtual Try-on in the Wild](https://mlsanity.com/paper/2403.05139/) (2024)
- [Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency](https://mlsanity.com/paper/2409.02634/) (2024)
- [IMAGDressing-v1: Customizable Virtual Dressing](https://mlsanity.com/paper/2407.12705/) (2024)
- [Video Interpolation with Diffusion Models](https://mlsanity.com/paper/2404.01203/) (2024)
- [Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks](https://mlsanity.com/paper/2501.15891/) (2025)
- [A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)](https://mlsanity.com/paper/2404.00579/) (2024)
- [CAT-DM: Controllable Accelerated Virtual Try-on with Diffusion Model](https://mlsanity.com/paper/2311.18405/) (2023)
- [Wear-Any-Way: Manipulable Virtual Try-on via Sparse Correspondence Alignment](https://mlsanity.com/paper/2403.12965/) (2024)
- [ViViD: Video Virtual Try-on using Diffusion Models](https://mlsanity.com/paper/2405.11794/) (2024)
- [CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attention](https://mlsanity.com/paper/2409.01876/) (2024)
- [Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On](https://mlsanity.com/paper/2404.01089/) (2024)
- [Street TryOn: Learning In-the-Wild Virtual Try-On from Unpaired Person Images](https://mlsanity.com/paper/2311.16094/) (2023)
- [Tunnel Try-on: Excavating Spatial-temporal Tunnels for High-quality Virtual Try-on in Videos](https://mlsanity.com/paper/2404.17571/) (2024)
- [OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulation](https://mlsanity.com/paper/2508.19209/) (2025)
- [SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generation](https://mlsanity.com/paper/2308.10156/) (2023)
- [A Survey on Personalized Content Synthesis with Diffusion Models](https://mlsanity.com/paper/2405.05538/) (2024)
- [M&M VTO: Multi-Garment Virtual Try-On and Editing](https://mlsanity.com/paper/2406.04542/) (2024)
- [StableGarment: Garment-Centric Generation via Stable Diffusion](https://mlsanity.com/paper/2403.10783/) (2024)
- [Learning Flow Fields in Attention for Controllable Person Image Generation](https://mlsanity.com/paper/2412.08486/) (2024)
- [MV-VTON: Multi-View Virtual Try-On with Diffusion Models](https://mlsanity.com/paper/2404.17364/) (2024)
- [MMTryon: Multi-Modal Multi-Reference Control for High-Quality Fashion Generation](https://mlsanity.com/paper/2405.00448/) (2024)
- [AnyFit: Controllable Virtual Try-on for Any Combination of Attire Across Any Scenario](https://mlsanity.com/paper/2405.18172/) (2024)
- [WildVidFit: Video Virtual Try-On in the Wild via Image-Based Controlled Diffusion Models](https://mlsanity.com/paper/2407.10625/) (2024)
- [SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation](https://mlsanity.com/paper/2411.18229/) (2024)

## Abstract (arXiv)

Given two images depicting a person and a garment worn by another person, our goal is to generate a visualization of how the garment might look on the input person. A key challenge is to synthesize a photorealistic detail-preserving visualization of the garment, while warping the garment to accommodate a significant body pose and shape change across the subjects. Previous methods either focus on garment detail preservation without effective pose and shape variation, or allow try-on with the desired shape and pose but lack garment details. In this paper, we propose a diffusion-based architecture that unifies two UNets (referred to as Parallel-UNet), which allows us to preserve garment details and warp the garment for significant pose and body change in a single network. The key ideas behind Parallel-UNet include: 1) garment is warped implicitly via a cross attention mechanism, 2) garment warp and person blend happen as part of a unified process as opposed to a sequence of two separate tasks. Experimental results indicate that TryOnDiffusion achieves state-of-the-art performance both qualitatively and quantitatively.