# Garment3DGen: 3D Garment Stylization and Texture Generation

arXiv: https://arxiv.org/abs/2403.18816 · HTML: https://mlsanity.com/paper/2403.18816/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 111 references that are in the map.
- First posted on arXiv 27 Mar 2024.

## Builds on (most cited first, 30 of 70)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model](https://mlsanity.com/paper/2310.15110/) (2023)
- [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors](https://mlsanity.com/paper/2306.17843/) (2023)
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [CLIP-Mesh: Generating textured meshes from text using pretrained image-text models](https://mlsanity.com/paper/2203.13333/) (2022)
- [Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack](https://mlsanity.com/paper/2309.15807/) (2023)
- [Demystifying CLIP Data](https://mlsanity.com/paper/2309.16671/) (2023)
- [Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers](https://mlsanity.com/paper/2312.09147/) (2023)
- [Text2Tex: Text-driven Texture Synthesis via Diffusion Models](https://mlsanity.com/paper/2303.11396/) (2023)
- [Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings](https://mlsanity.com/paper/1803.08495/) (2018)
- [SMPLicit: Topology-aware Generative Model for Clothed People](https://mlsanity.com/paper/2103.06871/) (2021)
- [Flexible Isosurface Extraction for Gradient-Based Mesh Optimization](https://mlsanity.com/paper/2308.05371/) (2023)
- [Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models](https://mlsanity.com/paper/2312.13913/) (2023)
- [BCNet: Learning Body and Cloth Shape from A Single Image](https://mlsanity.com/paper/2004.00214/) (2020)
- [Neural Cages for Detail-Preserving 3D Deformations](https://mlsanity.com/paper/1912.06395/) (2019)
- [TextDeformer: Geometry Manipulation using Text Guidance](https://mlsanity.com/paper/2304.13348/) (2023)
- [Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fields](https://mlsanity.com/paper/2207.13807/) (2022)

## Built on by (most cited first, 26 of 26)

- [ChatGarment: Garment Estimation, Generation and Editing via Large Language Models](https://mlsanity.com/paper/2412.17811/) (2024)
- [GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details](https://mlsanity.com/paper/2405.12420/) (2024)
- [3D Mesh Editing using Masked LRMs](https://mlsanity.com/paper/2412.08641/) (2024)
- [GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting](https://mlsanity.com/paper/2410.05259/) (2024)
- [Make-A-Texture: Fast Shape-Aware Texture Generation in 3 Seconds](https://mlsanity.com/paper/2412.07766/) (2024)
- [Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation](https://mlsanity.com/paper/2412.14453/) (2024)
- [GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing](https://mlsanity.com/paper/2503.08678/) (2025)
- [GarmageNet: A Multimodal Generative Framework for Sewing Pattern Design and Generic Garment Modeling](https://mlsanity.com/paper/2504.01483/) (2025)
- [WaSt-3D: Wasserstein-2 Distance for Scene-to-Scene Stylization on 3D Gaussians](https://mlsanity.com/paper/2409.17917/) (2024)
- [GarmentX: Autoregressive Parametric Representations for High-Fidelity 3D Garment Generation](https://mlsanity.com/paper/2504.20409/) (2025)
- [CRAFT: Designing Creative and Functional 3D Objects](https://mlsanity.com/paper/2412.03889/) (2024)
- [A Text-to-3D Framework for Joint Generation of CG-Ready Humans and Compatible Garments](https://mlsanity.com/paper/2503.12052/) (2025)
- [D-Garment: Physically Grounded Latent Diffusion for Dynamic Garment Deformations](https://mlsanity.com/paper/2504.03468/) (2025)
- [NI-Tex: Non-isometric Image-based Garment Texture Generation](https://mlsanity.com/paper/2511.18765/) (2025)
- [DressWild: Feed-Forward Pose-Agnostic Garment Sewing Pattern Generation from In-the-Wild Images](https://mlsanity.com/paper/2602.16502/) (2026)
- [AssetGen: Deployable 3D Asset Generation at Interactive Speed](https://mlsanity.com/paper/2605.26137/) (2026)
- [A Comparative Study of Garment Draping Techniques](https://mlsanity.com/paper/2405.11056/) (2024)
- [FabricDiffusion: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Clothing Images](https://mlsanity.com/paper/2410.01801/) (2024)
- [InverseDraping: Recovering Sewing Patterns from 3D Garment Surfaces via BoxMesh Bridging](https://mlsanity.com/paper/2604.02764/) (2026)
- [AnySurf: Any Surface Generation with Directed Edge](https://mlsanity.com/paper/2605.26149/) (2026)
- [Learning-based Seam Correspondence Reconstruction in Sewing Patterns](https://mlsanity.com/paper/2607.21213/) (2026)
- [GarmentPainter: Efficient 3D Garment Texture Synthesis with Character-Guided Diffusion Model](https://mlsanity.com/paper/2603.08228/) (2026)
- [SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representation](https://mlsanity.com/paper/2603.19053/) (2026)
- [HyperBones: Realtime Bone-driven Neural Garment Simulation with Hypernetwork Conditioning](https://mlsanity.com/paper/2605.20460/) (2026)
- [Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design](https://mlsanity.com/paper/2607.23189/) (2026)
- [OmniFabric: Coherent UV Space Texture Synthesis for 3D Garment Reconstruction](https://mlsanity.com/paper/2609.30234/) (2026)

## Abstract (arXiv)

We introduce Garment3DGen a new method to synthesize 3D garment assets from a base mesh given a single input image as guidance. Our proposed approach allows users to generate 3D textured clothes based on both real and synthetic images, such as those generated by text prompts. The generated assets can be directly draped and simulated on human bodies. We leverage the recent progress of image-to-3D diffusion methods to generate 3D garment geometries. However, since these geometries cannot be utilized directly for downstream tasks, we propose to use them as pseudo ground-truth and set up a mesh deformation optimization procedure that deforms a base template mesh to match the generated 3D target. Carefully designed losses allow the base mesh to freely deform towards the desired target, yet preserve mesh quality and topology such that they can be simulated. Finally, we generate high-fidelity texture maps that are globally and locally consistent and faithfully capture the input guidance, allowing us to render the generated 3D assets. With Garment3DGen users can generate the simulation-ready 3D garment of their choice without the need of artist intervention. We present a plethora of quantitative and qualitative comparisons on various assets and demonstrate that Garment3DGen unlocks key applications ranging from sketch-to-simulated garments or interacting with the garments in VR. Code is publicly available.