mlsanityData to 7 Oct 2026

FashionSD-X: Multimodal Fashion Garment Synthesis using Latent Diffusion

arXiv 2404.18591 ↗First posted 26 Apr 2024cs.CV cs.AIcited by 0

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

The rapid evolution of the fashion industry increasingly intersects with technological advancements, particularly through the integration of generative AI. This study introduces a novel generative pipeline designed to transform the fashion design process by employing latent diffusion models. Utilizing ControlNet and LoRA fine-tuning, our approach generates high-quality images from multimodal inputs such as text and sketches. We leverage and enhance state-of-the-art virtual try-on datasets, including Multimodal Dress Code and VITON-HD, by integrating sketch data. Our evaluation, utilizing metrics like FID, CLIP Score, and KID, demonstrates that our model significantly outperforms traditional stable diffusion models. The results not only highlight the effectiveness of our model in generating fashion-appropriate outputs but also underscore the potential of diffusion models in revolutionizing fashion design workflows. This research paves the way for more interactive, personalized, and technologically enriched methodologies in fashion design and representation, bridging the gap between creative vision and practical application.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 34 of its 39 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

34 of 39 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-On22 May
  2. 2023Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing4 Apr
  3. 2022Person Image Synthesis via Denoising Diffusion Model22 Nov

Other references · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
19 more
  1. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  3. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  4. 2018Demystifying MMD GANscited by 1,268
  5. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  6. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  7. 2021VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalizationcited by 287
  8. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  9. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  10. 2017GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibriumcited by 208
  11. 2022Sketch-Guided Text-to-Image Diffusion Modelscited by 180
  12. 2020DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesiscited by 161
  13. 2022Dress Code: High-Resolution Multi-Category Virtual Try-Oncited by 161
  14. 2019DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Imagescited by 132
  15. 2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51
  16. 2017Be Your Own Prada: Fashion Synthesis with Structural Coherencecited by 49
  17. 2022Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Modelcited by 39
  18. 2022Text2Human: Text-Driven Controllable Human Image Generationcited by 31
  19. 2023FICE: Text-Conditioned Fashion Image Editing With Guided GAN Inversioncited by 21

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing
  2. 2023Interactive Fashion Content Generation Using LLMs and Latent Diffusion Models
  3. 2024Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation
  4. 2025Cross-Cultural Fashion Design via Interactive Large Language Models and Diffusion Models
  5. 2024DPDEdit: Detail-Preserved Diffusion Models for Multimodal Fashion Image Editing
  6. 2025Fashion-RAG: Multimodal Fashion Image Editing via Retrieval-Augmented Generation
  7. 2023SGDiff: A Style Guided Diffusion Model for Fashion Synthesis
  8. 2025LOTS of Fashion! Multi-Conditioning for Image Generation via Sketch-Text Pairing