# GarmentDiffusion: 3D Garment Sewing Pattern Generation with Multimodal Diffusion Transformers arXiv: https://arxiv.org/abs/2504.21476 ยท HTML: https://mlsanity.com/paper/2504.21476/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. - Builds on 28 of its 43 references that are in the map. - First posted on arXiv 30 Apr 2025. ## Placed in - Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/) ## Builds on (most cited first, 28 of 28) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020) - [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022) - [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016) - [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023) - [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023) - [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023) - [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023) - [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020) - [BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometry](https://mlsanity.com/paper/2401.15563/) (2024) - [DeepCloth: Neural Garment Representation for Shape and Style Editing](https://mlsanity.com/paper/2011.14619/) (2020) - [Learning a Shared Shape Space for Multimodal Garment Design](https://mlsanity.com/paper/1806.11335/) (2018) - [DressCode: Autoregressively Sewing and Generating Garments from Text Guidance](https://mlsanity.com/paper/2401.16465/) (2024) - [3D Clothed Human Reconstruction in the Wild](https://mlsanity.com/paper/2207.10053/) (2022) - [Generating Datasets of 3D Garments with Sewing Patterns](https://mlsanity.com/paper/2109.05633/) (2021) - [Towards Garment Sewing Pattern Reconstruction from a Single Image](https://mlsanity.com/paper/2311.04218/) (2023) - [GarmentCode: Programming Parametric Sewing Patterns](https://mlsanity.com/paper/2306.03642/) (2023) - [Text2CAD: Generating Sequential CAD Models from Beginner-to-Expert Level Text Prompts](https://mlsanity.com/paper/2409.17106/) (2024) - [ChatGarment: Garment Estimation, Generation and Editing via Large Language Models](https://mlsanity.com/paper/2412.17811/) (2024) - [GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details](https://mlsanity.com/paper/2405.12420/) (2024) - [Structure-Preserving 3D Garment Modeling with Neural Sewing Machines](https://mlsanity.com/paper/2211.06701/) (2022) - [Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis](https://mlsanity.com/paper/2412.08603/) (2024) - [WordRobe: Text-Guided Generation of Textured 3D Garments](https://mlsanity.com/paper/2403.17541/) (2024) - [Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation](https://mlsanity.com/paper/2412.14453/) (2024) - [SPnet: Estimating Garment Sewing Patterns from a Single Image](https://mlsanity.com/paper/2312.16264/) (2023) - [Design2Cloth: 3D Cloth Generation from 2D Masks](https://mlsanity.com/paper/2404.02686/) (2024) ## Built on by (most cited first, 9 of 9) - [DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers](https://mlsanity.com/paper/2505.21541/) (2025) - [NGL: Natural Garment Language for Training-Free Sewing Pattern Estimation](https://mlsanity.com/paper/2602.20700/) (2026) - [Garment Particles: A 2D--3D Symmetric Garment Representation for Generation and Editing](https://mlsanity.com/paper/2605.26391/) (2026) - [Learning Sewing Patterns via Latent Flow Matching of Implicit Fields](https://mlsanity.com/paper/2601.17740/) (2026) - [PatternGSL: A Structured Specification Language for Template-Free and Simulation-Ready 3D Garments](https://mlsanity.com/paper/2606.24564/) (2026) - [SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representation](https://mlsanity.com/paper/2603.19053/) (2026) - [Stitched Embeddings: A Unified Latent Space for 3D Garments and 2D Patterns](https://mlsanity.com/paper/2607.00829/) (2026) - [TailorCoPilot: Enabling Agentic Pattern Making with Version-Controlled State Tracking](https://mlsanity.com/paper/2608.25462/) (2026) - [SewFusion: Tailored Generation of Topology and Panel-Level Geometry for Sewing Patterns](https://mlsanity.com/paper/2609.23548/) (2026) ## Abstract (arXiv) Garment sewing patterns are fundamental design elements that bridge the gap between design concepts and practical manufacturing. The generative modeling of sewing patterns is crucial for creating diversified garments. However, existing approaches are limited either by reliance on a single input modality or by suboptimal generation efficiency. In this work, we present GarmentDiffusion, a new generative model capable of producing centimeter-precise, vectorized 3D sewing patterns from multimodal inputs (text, image, and incomplete sewing pattern). Our method efficiently encodes 3D sewing pattern parameters into compact edge token representations, achieving a sequence length that is 10 times shorter than that of the autoregressive SewingGPT in DressCode. By employing a diffusion transformer, we simultaneously denoise all edge tokens along the temporal axis, while maintaining a constant number of denoising steps regardless of dataset-specific edge and panel statistics. With all combination of designs of our model, the sewing pattern generation speed is accelerated by 100 times compared to SewingGPT. We achieve new state-of-the-art results on DressCodeData, as well as on the largest sewing pattern dataset, namely GarmentCodeData. The project website is available at https://shenfu-research.github.io/Garment-Diffusion/.