# Structure-Preserving 3D Garment Modeling with Neural Sewing Machines

arXiv: https://arxiv.org/abs/2211.06701 · HTML: https://mlsanity.com/paper/2211.06701/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 12 Nov 2022.

## Builds on (most cited first, 30 of 32)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019)
- [Learning to Reconstruct People in Clothing from a Single RGB Camera](https://mlsanity.com/paper/1903.05885/) (2019)
- [SMPLicit: Topology-aware Generative Model for Clothed People](https://mlsanity.com/paper/2103.06871/) (2021)
- [TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style](https://mlsanity.com/paper/2003.04583/) (2020)
- [Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction](https://mlsanity.com/paper/2007.11432/) (2020)
- [BCNet: Learning Body and Cloth Shape from A Single Image](https://mlsanity.com/paper/2004.00214/) (2020)
- [CLOTH3D: Clothed 3D Humans](https://mlsanity.com/paper/1912.02792/) (2019)
- [Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images](https://mlsanity.com/paper/2003.12753/) (2020)
- [DeepWrinkles: Accurate and Realistic Clothing Modeling](https://mlsanity.com/paper/1808.03417/) (2018)
- [SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing](https://mlsanity.com/paper/2007.11610/) (2020)
- [Graphonomy: Universal Human Parsing via Graph Transfer Learning](https://mlsanity.com/paper/1904.04536/) (2019)
- [360-Degree Textures of People in Clothing from a Single Image](https://mlsanity.com/paper/1908.07117/) (2019)
- [MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video](https://mlsanity.com/paper/2009.10711/) (2020)
- [DeepCloth: Neural Garment Representation for Shape and Style Editing](https://mlsanity.com/paper/2011.14619/) (2020)
- [Learning a Shared Shape Space for Multimodal Garment Design](https://mlsanity.com/paper/1806.11335/) (2018)
- [Learning Implicit Templates for Point-Based Clothed Human Modeling](https://mlsanity.com/paper/2207.06955/) (2022)
- [Fully Convolutional Graph Neural Networks for Parametric Virtual Try-On](https://mlsanity.com/paper/2009.04592/) (2020)
- [Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks](https://mlsanity.com/paper/2205.01355/) (2022)
- [Learning Anchored Unsigned Distance Functions with Gradient Direction Alignment for Single-view Garment Reconstruction](https://mlsanity.com/paper/2108.08478/) (2021)
- [Generating Datasets of 3D Garments with Sewing Patterns](https://mlsanity.com/paper/2109.05633/) (2021)
- [M3D-VTON: A Monocular-to-3D Virtual Try-On Network](https://mlsanity.com/paper/2108.05126/) (2021)
- [Dynamic Neural Garments](https://mlsanity.com/paper/2102.11811/) (2021)
- [Garment4D: Garment Reconstruction from Point Cloud Sequences](https://mlsanity.com/paper/2112.04159/) (2021)
- [Deep Detail Enhancement for Any Garment](https://mlsanity.com/paper/2008.04367/) (2020)
- [Detailed Garment Recovery from a Single-View Image](https://mlsanity.com/paper/1608.01250/) (2016)

## Built on by (most cited first, 18 of 18)

- [SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2303.16196/) (2023)
- [DressCode: Autoregressively Sewing and Generating Garments from Text Guidance](https://mlsanity.com/paper/2401.16465/) (2024)
- [DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation](https://mlsanity.com/paper/2311.12194/) (2023)
- [Towards Garment Sewing Pattern Reconstruction from a Single Image](https://mlsanity.com/paper/2311.04218/) (2023)
- [ISP: Multi-Layered Garment Draping with Implicit Sewing Patterns](https://mlsanity.com/paper/2305.14100/) (2023)
- [Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis](https://mlsanity.com/paper/2412.08603/) (2024)
- [ClotheDreamer: Text-Guided Garment Generation with 3D Gaussians](https://mlsanity.com/paper/2406.16815/) (2024)
- [GarmentDiffusion: 3D Garment Sewing Pattern Generation with Multimodal Diffusion Transformers](https://mlsanity.com/paper/2504.21476/) (2025)
- [GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing](https://mlsanity.com/paper/2503.08678/) (2025)
- [GarmageNet: A Multimodal Generative Framework for Sewing Pattern Design and Generic Garment Modeling](https://mlsanity.com/paper/2504.01483/) (2025)
- [DressWild: Feed-Forward Pose-Agnostic Garment Sewing Pattern Generation from In-the-Wild Images](https://mlsanity.com/paper/2602.16502/) (2026)
- [NGL: Natural Garment Language for Training-Free Sewing Pattern Estimation](https://mlsanity.com/paper/2602.20700/) (2026)
- [FabricDiffusion: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Clothing Images](https://mlsanity.com/paper/2410.01801/) (2024)
- [Learning Sewing Patterns via Latent Flow Matching of Implicit Fields](https://mlsanity.com/paper/2601.17740/) (2026)
- [InverseDraping: Recovering Sewing Patterns from 3D Garment Surfaces via BoxMesh Bridging](https://mlsanity.com/paper/2604.02764/) (2026)
- [Learning-based Seam Correspondence Reconstruction in Sewing Patterns](https://mlsanity.com/paper/2607.21213/) (2026)
- [Textile IR: A Bidirectional Intermediate Representation for Physics-Aware Fashion CAD](https://mlsanity.com/paper/2601.02792/) (2026)
- [SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representation](https://mlsanity.com/paper/2603.19053/) (2026)

## Abstract (arXiv)

3D Garment modeling is a critical and challenging topic in the area of computer vision and graphics, with increasing attention focused on garment representation learning, garment reconstruction, and controllable garment manipulation, whereas existing methods were constrained to model garments under specific categories or with relatively simple topologies. In this paper, we propose a novel Neural Sewing Machine (NSM), a learning-based framework for structure-preserving 3D garment modeling, which is capable of learning representations for garments with diverse shapes and topologies and is successfully applied to 3D garment reconstruction and controllable manipulation. To model generic garments, we first obtain sewing pattern embedding via a unified sewing pattern encoding module, as the sewing pattern can accurately describe the intrinsic structure and the topology of the 3D garment. Then we use a 3D garment decoder to decode the sewing pattern embedding into a 3D garment using the UV-position maps with masks. To preserve the intrinsic structure of the predicted 3D garment, we introduce an inner-panel structure-preserving loss, an inter-panel structure-preserving loss, and a surface-normal loss in the learning process of our framework. We evaluate NSM on the public 3D garment dataset with sewing patterns with diverse garment shapes and categories. Extensive experiments demonstrate that the proposed NSM is capable of representing 3D garments under diverse garment shapes and topologies, realistically reconstructing 3D garments from 2D images with the preserved structure, and accurately manipulating the 3D garment categories, shapes, and topologies, outperforming the state-of-the-art methods by a clear margin.