# MMTryon: Multi-Modal Multi-Reference Control for High-Quality Fashion Generation

arXiv: https://arxiv.org/abs/2405.00448 · HTML: https://mlsanity.com/paper/2405.00448/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 57 references that are in the map.
- First posted on arXiv 1 May 2024.

## Builds on (most cited first, 30 of 47)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023)
- [Composer: Creative and Controllable Image Synthesis with Composable Conditions](https://mlsanity.com/paper/2302.09778/) (2023)
- [VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization](https://mlsanity.com/paper/2103.16874/) (2021)
- [Toward Characteristic-Preserving Image-based Virtual Try-On Network](https://mlsanity.com/paper/1807.07688/) (2018)
- [Self-Correction for Human Parsing](https://mlsanity.com/paper/1910.09777/) (2019)
- [StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On](https://mlsanity.com/paper/2312.01725/) (2023)
- [Dress Code: High-Resolution Multi-Category Virtual Try-On](https://mlsanity.com/paper/2204.08532/) (2022)
- [LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-On](https://mlsanity.com/paper/2305.13501/) (2023)
- [OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on](https://mlsanity.com/paper/2403.01779/) (2024)
- [Towards Photo-Realistic Virtual Try-On by Adaptively GeneratingleftrightarrowPreserving Image Content](https://mlsanity.com/paper/2003.05863/) (2020)
- [Parser-Free Virtual Try-on via Distilling Appearance Flows](https://mlsanity.com/paper/2103.04559/) (2021)
- [TryOnDiffusion: A Tale of Two UNets](https://mlsanity.com/paper/2306.08276/) (2023)

## Built on by (most cited first, 23 of 23)

- [CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models](https://mlsanity.com/paper/2407.15886/) (2024)
- [Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks](https://mlsanity.com/paper/2501.15891/) (2025)
- [CatV2TON: Taming Diffusion Transformers for Vision-Based Virtual Try-On with Temporal Concatenation](https://mlsanity.com/paper/2501.11325/) (2025)
- [AnyFit: Controllable Virtual Try-on for Any Combination of Attire Across Any Scenario](https://mlsanity.com/paper/2405.18172/) (2024)
- [OutfitAnyone: Ultra-high Quality Virtual Try-On for Any Clothing and Any Person](https://mlsanity.com/paper/2407.16224/) (2024)
- [VTON-HandFit: Virtual Try-on for Arbitrary Hand Pose Guided by Hand Priors Embedding](https://mlsanity.com/paper/2408.12340/) (2024)
- [Pursuing Temporal-Consistent Video Virtual Try-On via Dynamic Pose Interaction](https://mlsanity.com/paper/2505.16980/) (2025)
- [FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable Diffusion Models](https://mlsanity.com/paper/2508.20586/) (2025)
- [DreamFit: Garment-Centric Human Generation via a Lightweight Anything-Dressing Encoder](https://mlsanity.com/paper/2412.17644/) (2024)
- [Controllable Human Image Generation with Personalized Multi-Garments](https://mlsanity.com/paper/2411.16801/) (2024)
- [Garments2Look: A Multi-Reference Dataset for High-Fidelity Outfit-Level Virtual Try-On with Clothing and Accessories](https://mlsanity.com/paper/2603.14153/) (2026)
- [UniFit: Towards Universal Virtual Try-on with MLLM-Guided Semantic Alignment](https://mlsanity.com/paper/2511.15831/) (2025)
- [DiT-VTON: Diffusion Transformer Framework for Unified Multi-Category Virtual Try-On and Virtual Try-All with Integrated Image Editing](https://mlsanity.com/paper/2510.04797/) (2025)
- [RoomEditor++: A Parameter-Sharing Diffusion Architecture for High-Fidelity Furniture Synthesis](https://mlsanity.com/paper/2512.17573/) (2025)
- [GO-MLVTON: Garment Occlusion-Aware Multi-Layer Virtual Try-On with Diffusion Models](https://mlsanity.com/paper/2601.13524/) (2026)
- [TalkFashion: Intelligent Virtual Try-On Assistant Based on Multimodal Large Language Model](https://mlsanity.com/paper/2507.05790/) (2025)
- [Two-Way Garment Transfer: Unified Diffusion Framework for Dressing and Undressing Synthesis](https://mlsanity.com/paper/2508.04551/) (2025)
- [Efficient Encoder-Free Pose Conditioning and Pose Control for Virtual Try-On](https://mlsanity.com/paper/2509.20343/) (2025)
- [FitVTON: Fit-aware Virtual Try-On via Body-Garment Size Control](https://mlsanity.com/paper/2606.12012/) (2026)
- [WearWow: Native 2K Multi-Garment Virtual Try-On via Adaptive Token Packing and Preference Alignment](https://mlsanity.com/paper/2607.19923/) (2026)
- [Layering Virtual Try-On](https://mlsanity.com/paper/2607.22924/) (2026)
- [TexTailor: Texture-Preserving Video Virtual Try-On via Adaptive Garment Conditioning](https://mlsanity.com/paper/2609.39335/) (2026)
- [ShoeFit: A New Dataset and Dual-image-stream DiT Framework for Virtual Footwear Try-On](https://mlsanity.com/paper/2610.18325/) (2026)

## Abstract (arXiv)

This paper introduces MMTryon, a multi-modal multi-reference VIrtual Try-ON (VITON) framework, which can generate high-quality compositional try-on results by taking a text instruction and multiple garment images as inputs. Our MMTryon addresses three problems overlooked in prior literature: 1) Support of multiple try-on items. Existing methods are commonly designed for single-item try-on tasks (e.g., upper/lower garments, dresses). 2) Specification of dressing style. Existing methods are unable to customize dressing styles based on instructions (e.g., zipped/unzipped, tuck-in/tuck-out, etc.) 3) Segmentation Dependency. They further heavily rely on category-specific segmentation models to identify the replacement regions, with segmentation errors directly leading to significant artifacts in the try-on results. To address the first two issues, our MMTryon introduces a novel multi-modality and multi-reference attention mechanism to combine the garment information from reference images and dressing-style information from text instructions. Besides, to remove the segmentation dependency, MMTryon uses a parsing-free garment encoder and leverages a novel scalable data generation pipeline to convert existing VITON datasets to a form that allows MMTryon to be trained without requiring any explicit segmentation. Extensive experiments on high-resolution benchmarks and in-the-wild test sets demonstrate MMTryon's superiority over existing SOTA methods both qualitatively and quantitatively. MMTryon's impressive performance on multi-item and style-controllable virtual try-on scenarios and its ability to try on any outfit in a large variety of scenarios from any source image, opens up a new avenue for future investigation in the fashion community.