mlsanityData to 14 Oct 2026

Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing

arXiv 2304.02051 ↗First posted 4 Apr 2023cs.CV cs.AI cs.MMcited by 62

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Fashion illustration is used by designers to communicate their vision and to bring the design idea from conceptualization to realization, showing how clothes interact with the human body. In this context, computer vision can thus be used to improve the fashion design process. Differently from previous works that mainly focused on the virtual try-on of garments, we propose the task of multimodal-conditioned fashion image editing, guiding the generation of human-centric fashion images by following multimodal prompts, such as text, human body poses, and garment sketches. We tackle this problem by proposing a new architecture based on latent diffusion models, an approach that has not been used before in the fashion domain. Given the lack of existing datasets suitable for the task, we also extend two existing fashion datasets, namely Dress Code and VITON-HD, with multimodal annotations collected in a semi-automatic manner. Experimental results on these new datasets demonstrate the effectiveness of our proposal, both in terms of realism and coherence with the given multimodal inputs. Source code and collected multimodal annotations are publicly available at: https://github.com/aimagelab/multimodal-garment-designer.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 51 of its 61 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

51 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan

Other references · 50

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2020Denoising Diffusion Implicit Modelscited by 9,781
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
38 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  4. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  5. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  6. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  7. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  8. 2018Demystifying MMD GANscited by 1,293
  9. 2017Mixed Precision Trainingcited by 1,156
  10. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  11. 2021Robust fine-tuning of zero-shot modelscited by 797
  12. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  13. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  14. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  15. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  16. 2021VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalizationcited by 287
  17. 2017VITON: An Image-based Virtual Try-on Networkcited by 286
  18. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  19. 2022MAT: Mask-Aware Transformer for Large Hole Image Inpaintingcited by 218
  20. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  21. 2017GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibriumcited by 213
  22. 2018Toward Characteristic-Preserving Image-based Virtual Try-On Networkcited by 192
  23. 2022Sketch-Guided Text-to-Image Diffusion Modelscited by 180
  24. 2021Pixel Difference Networks for Efficient Edge Detectioncited by 168
  25. 2020DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesiscited by 162
  26. 2022Dress Code: High-Resolution Multi-Category Virtual Try-Oncited by 162
  27. 2022Pretraining is All You Need for Image-to-Image Translationcited by 160
  28. 2020Towards Photo-Realistic Virtual Try-On by Adaptively GeneratingleftrightarrowPreserving Image Contentcited by 149
  29. 2017Convolutional neural network architecture for geometric matchingcited by 138
  30. 2017Automatic Spatially-aware Fashion Concept Discoverycited by 134
  31. 2021OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Associationcited by 55
  32. 2017Be Your Own Prada: Fashion Synthesis with Structural Coherencecited by 49
  33. 2022Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Modelcited by 39
  34. 2019Context-Aware Visual Compatibility Predictioncited by 26
  35. 2017Creating Capsule Wardrobes from Fashion Imagescited by 24
  36. 2022OutfitTransformer: Learning Outfit Representations for Fashion Recommendationcited by 24
  37. 2023FICE: Text-Conditioned Fashion Image Editing With Guided GAN Inversioncited by 21
  38. 2021Query2Prod2Vec Grounded Word Embeddings for eCommercecited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 29; 2025: 14; 2026: 10.

62 papers.

Sort:

By area:

  1. 2023LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-Oncited by 152
  2. 2023Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Modelscited by 104
  3. 2023CAT-DM: Controllable Accelerated Virtual Try-on with Diffusion Modelcited by 48
  4. 2024ViViD: Video Virtual Try-on using Diffusion Modelscited by 42
  5. 2024Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-Oncited by 38
  6. 2024Tunnel Try-on: Excavating Spatial-temporal Tunnels for High-quality Virtual Try-on in Videoscited by 32
  7. 2023Image-Based Virtual Try-On: A Surveycited by 19
  8. 2024UniFashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generationcited by 17
  9. 2023PICTURE: PhotorealistIC virtual Try-on from UnconstRained dEsignscited by 14
  10. 2024Giving a Hand to Diffusion Models: a Two-Stage Approach to Improving Conditional Human Image Generationcited by 13
  11. 2024Multimodal Latent Diffusion Model for Complex Sewing Pattern Generationcited by 12
  12. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
38 more
  1. 2023OpenFashionCLIP: Vision-and-Language Contrastive Learning with Open-Source Fashion Datacited by 9
  2. 2024AdaptiveDrag: Semantic-Driven Dragging on Diffusion-Based Image Editingcited by 8
  3. 2025Voost: A Unified and Scalable Diffusion Transformer for Bidirectional Virtual Try-On and Try-Offcited by 8
  4. 2025LOTS of Fashion! Multi-Conditioning for Image Generation via Sketch-Text Pairingcited by 7
  5. 2024HieraFashDiff: Hierarchical Fashion Design with Multi-stage Diffusion Modelscited by 6
  6. 2025MGT: Extending Virtual Try-Off to Multi-Garment Scenarioscited by 6
  7. 2023Learning-To-Rank Approach for Identifying Everyday Objects Using a Physical-World Search Enginecited by 5
  8. 2024Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editingcited by 5
  9. 2025StyleTailor: Towards Personalized Fashion Styling via Hierarchical Negative Feedbackcited by 5
  10. 2023FIRST: A Million-Entry Dataset for Text-Driven Fashion Synthesis and Designcited by 4
  11. 2024Better Fit: Accommodate Variations in Clothing Types for Virtual Try-oncited by 4
  12. 2025Inverse Virtual Try-On: Generating Multi-Category Product-Style Images from Clothed Individualscited by 4
  13. 2024PFDM: Parser-Free Virtual Try-on via Diffusion Modelcited by 3
  14. 2024TryOn-Adapter: Efficient Fine-Grained Clothing Identity Adaptation for High-Fidelity Virtual Try-Oncited by 3
  15. 2025Enhancing Virtual Try-On with Synthetic Pairs and Error-Aware Noise Schedulingcited by 3
  16. 2025Video Virtual Try-on with Conditional Diffusion Transformer Inpaintercited by 3
  17. 2026DetailAnywhere: Fashion Detail Generation via Cross-Modal Feature Alignment Distillationcited by 3
  18. 2024GroupDiff: Diffusion-based Group Portrait Editingcited by 2
  19. 2024SAT-LDM: Provably Generalizable Image Watermarking for Latent Diffusion Models with Self-Augmented Trainingcited by 2
  20. 2025Identity-Preserving Text-to-Image Generation via Dual-Level Feature Decoupling and Expert-Guided Fusioncited by 2
  21. 2025DualFit: A Two-Stage Virtual Try-On via Warping and Synthesiscited by 2
  22. 2023StyleHumanCLIP: Text-guided Garment Manipulation for StyleGAN-Humancited by 1
  23. 2024Sketch2Human: Deep Human Generation with Disentangled Geometry and Appearance Controlcited by 1
  24. 2024TexControl: Sketch-Based Two-Stage Fashion Image Generation Using Diffusion Modelcited by 1
  25. 2024Masked Extended Attention for Zero-Shot Virtual Try-On In The Wildcited by 1
  26. 2024DiCTI: Diffusion-based Clothing Designer via Text-guided Inputcited by 1
  27. 2024AnyDesign: Versatile Area Fashion Editing via Mask-Free Diffusioncited by 1
  28. 2024DH-VTON: Deep Text-Driven Virtual Try-On via Hybrid Attention Learningcited by 1
  29. 2024Dressing the Imagination: A Dataset for AI-Powered Translation of Text into Fashion Outfits and A Novel NeRA Adapter for Enhanced Feature Adaptationcited by 1
  30. 2025Fashion-RAG: Multimodal Fashion Image Editing via Retrieval-Augmented Generationcited by 1
  31. 2025Shape-Guided Clothing Warping for Virtual Try-Oncited by 1
  32. 2025ChronoTailor: Harnessing Attention Guidance for Fine-Grained Video Virtual Try-Oncited by 1
  33. 2026OmniDiT: Extending Diffusion Transformer to Omni-VTON Frameworkcited by 1
  34. 2024FashionSD-X: Multimodal Fashion Garment Synthesis using Latent Diffusioncited by 0
  35. 2024GraVITON: Graph based garment warping with attention guided inversion for Virtual-tryoncited by 0
  36. 2024DPDEdit: Detail-Preserved Diffusion Models for Multimodal Fashion Image Editingcited by 0
  37. 2024Garment Attribute Manipulation with Multi-level Attentioncited by 0
  38. 2024ControlEdit: A MultiModal Local Clothing Image Editing Methodcited by 0

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024FashionEngine: Interactive 3D Human Generation and Editing via Multimodal Controls
  2. 2025HiGarment: Cross-modal Harmony Based Diffusion Model for Flat Sketch to Realistic Garment Image
  3. 2024DiffusionTrend: A Minimalist Approach to Virtual Fashion Try-On
  4. 2023DiffFashion: Reference-based Fashion Design with Structure-aware Transfer by Diffusion Models
  5. 2025MuGa-VTON: Multi-Garment Virtual Try-On via Diffusion Transformers with Prompt Customization
  6. 2023SGDiff: A Style Guided Diffusion Model for Fashion Synthesis
  7. 2024Improving Virtual Try-On with Garment-focused Diffusion Models
  8. 2023Interactive Fashion Content Generation Using LLMs and Latent Diffusion Models