# Sketch2Cloth: Sketch-based 3D Garment Generation with Unsigned Distance Fields

arXiv: https://arxiv.org/abs/2303.00167 · HTML: https://mlsanity.com/paper/2303.00167/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 36 references that are in the map.
- First posted on arXiv 1 Mar 2023.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Neural Unsigned Distance Fields for Implicit Function Learning](https://mlsanity.com/paper/2010.13938/) (2020)
- [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019)
- [Tex2Shape: Detailed Full Human Body Geometry From a Single Image](https://mlsanity.com/paper/1904.08645/) (2019)
- [MeshSDF: Differentiable Iso-Surface Extraction](https://mlsanity.com/paper/2006.03997/) (2020)
- [Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images](https://mlsanity.com/paper/2003.12753/) (2020)
- [DeepWrinkles: Accurate and Realistic Clothing Modeling](https://mlsanity.com/paper/1808.03417/) (2018)
- [MeshUDF: Fast and Differentiable Meshing of Unsigned Distance Field Networks](https://mlsanity.com/paper/2111.14549/) (2021)
- [NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies](https://mlsanity.com/paper/2211.14173/) (2022)
- [Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketches](https://mlsanity.com/paper/2105.06663/) (2021)
- [Learning a Shared Shape Space for Multimodal Garment Design](https://mlsanity.com/paper/1806.11335/) (2018)
- [Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches](https://mlsanity.com/paper/2104.00482/) (2021)
- [3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks](https://mlsanity.com/paper/1707.06375/) (2017)
- [Sketch-based 3D Shape Retrieval using Convolutional Neural Networks](https://mlsanity.com/paper/1504.03504/) (2015)
- [3D Sketching using Multi-View Deep Volumetric Prediction](https://mlsanity.com/paper/1707.08390/) (2017)
- [Learning Anchored Unsigned Distance Functions with Gradient Direction Alignment for Single-view Garment Reconstruction](https://mlsanity.com/paper/2108.08478/) (2021)
- [dualFace:Two-Stage Drawing Guidance for Freehand Portrait Sketching](https://mlsanity.com/paper/2104.12297/) (2021)
- [DualSmoke: Sketch-Based Smoke Illustration Design with Two-Stage Generative Model](https://mlsanity.com/paper/2208.10906/) (2022)
- [DiffFaceSketch: High-Fidelity Face Image Synthesis with Sketch-Guided Latent Diffusion Model](https://mlsanity.com/paper/2302.06908/) (2023)
- [Sketch-based Normal Map Generation with Geometric Sampling](https://mlsanity.com/paper/2104.11554/) (2021)
- [DualMotion: Global-to-Local Casual Motion Design for Character Animations](https://mlsanity.com/paper/2208.08636/) (2022)

## Built on by (most cited first, 5 of 5)

- [GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details](https://mlsanity.com/paper/2405.12420/) (2024)
- [Sketch2NeRF: Multi-view Sketch-guided Text-to-3D Generation](https://mlsanity.com/paper/2401.14257/) (2024)
- [TexControl: Sketch-Based Two-Stage Fashion Image Generation Using Diffusion Model](https://mlsanity.com/paper/2405.04675/) (2024)
- [From Air to Wear: Personalized 3D Digital Fashion with AR/VR Immersive 3D Sketching](https://mlsanity.com/paper/2505.09998/) (2025)
- [Stitched Embeddings: A Unified Latent Space for 3D Garments and 2D Patterns](https://mlsanity.com/paper/2607.00829/) (2026)

## Abstract (arXiv)

3D model reconstruction from a single image has achieved great progress with the recent deep generative models. However, the conventional reconstruction approaches with template mesh deformation and implicit fields have difficulty in reconstructing non-watertight 3D mesh models, such as garments. In contrast to image-based modeling, the sketch-based approach can help users generate 3D models to meet the design intentions from hand-drawn sketches. In this study, we propose Sketch2Cloth, a sketch-based 3D garment generation system using the unsigned distance fields from the user's sketch input. Sketch2Cloth first estimates the unsigned distance function of the target 3D model from the sketch input, and extracts the mesh from the estimated field with Marching Cubes. We also provide the model editing function to modify the generated mesh. We verified the proposed Sketch2Cloth with quantitative evaluations on garment generation and editing with a state-of-the-art approach.