# Neural Clothing Tryer: Customized Virtual Try-On via Semantic Enhancement and Controlling Diffusion Model

arXiv: https://arxiv.org/abs/2601.22838 · HTML: https://mlsanity.com/paper/2601.22838/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 32 of its 55 references that are in the map.
- First posted on arXiv 30 Jan 2026.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 32)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017)
- [MirrorGAN: Learning Text-to-image Generation by Redescription](https://mlsanity.com/paper/1903.05854/) (2019)
- [Dress Code: High-Resolution Multi-Category Virtual Try-On](https://mlsanity.com/paper/2204.08532/) (2022)
- [LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-On](https://mlsanity.com/paper/2305.13501/) (2023)
- [TryOnDiffusion: A Tale of Two UNets](https://mlsanity.com/paper/2306.08276/) (2023)
- [High-Resolution Virtual Try-On with Misalignment and Occlusion-Handled Conditions](https://mlsanity.com/paper/2206.14180/) (2022)
- [Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flow](https://mlsanity.com/paper/2308.06101/) (2023)
- [GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learning](https://mlsanity.com/paper/2303.13756/) (2023)
- [Object-driven Text-to-Image Synthesis via Adversarial Training](https://mlsanity.com/paper/1902.10740/) (2019)
- [Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing](https://mlsanity.com/paper/2304.02051/) (2023)
- [DiffFashion: Reference-based Fashion Design with Structure-aware Transfer by Diffusion Models](https://mlsanity.com/paper/2302.06826/) (2023)
- [C-VTON: Context-Driven Image-Based Virtual Try-On Network](https://mlsanity.com/paper/2212.04437/) (2022)
- [ClothFormer:Taming Video Virtual Try-on in All Module](https://mlsanity.com/paper/2204.12151/) (2022)
- [Heterogeneous Semantic Transfer for Multi-label Recognition with Partial Labels](https://mlsanity.com/paper/2205.11131/) (2022)
- [Dance with You: The Diversity Controllable Dancer Generation via Diffusion Models](https://mlsanity.com/paper/2308.13551/) (2023)
- [Dynamic Correlation Learning and Regularization for Multi-Label Confidence Calibration](https://mlsanity.com/paper/2407.06844/) (2024)
- [SPG-VTON: Semantic Prediction Guidance for Multi-pose Virtual Try-on](https://mlsanity.com/paper/2108.01578/) (2021)
- [OccluMix: Towards De-Occlusion Virtual Try-on by Semantically-Guided Mixup](https://mlsanity.com/paper/2301.00965/) (2023)
- [PG-VTON: A Novel Image-Based Virtual Try-On Method via Progressive Inference Paradigm](https://mlsanity.com/paper/2304.08956/) (2023)
- [Contrastive Decoupled Representation Learning and Regularization for Speech-Preserving Facial Expression Manipulation](https://mlsanity.com/paper/2504.05672/) (2025)

## Abstract (arXiv)

This work aims to address a novel Customized Virtual Try-ON (Cu-VTON) task, enabling the superimposition of a specified garment onto a model that can be customized in terms of appearance, posture, and additional attributes. Compared with traditional VTON task, it enables users to tailor digital avatars to their individual preferences, thereby enhancing the virtual fitting experience with greater flexibility and engagement. To address this task, we introduce a Neural Clothing Tryer (NCT) framework, which exploits the advanced diffusion models equipped with semantic enhancement and controlling modules to better preserve semantic characterization and textural details of the garment and meanwhile facilitating the flexible editing of the model's postures and appearances. Specifically, NCT introduces a semantic-enhanced module to take semantic descriptions of garments and utilizes a visual-language encoder to learn aligned features across modalities. The aligned features are served as condition input to the diffusion model to enhance the preservation of the garment's semantics. Then, a semantic controlling module is designed to take the garment image, tailored posture image, and semantic description as input to maintain garment details while simultaneously editing model postures, expressions, and various attributes. Extensive experiments on the open available benchmark demonstrate the superior performance of the proposed NCT framework.