# Deep Sketch-Based Modeling: Tips and Tricks

arXiv: https://arxiv.org/abs/2011.06133 · HTML: https://mlsanity.com/paper/2011.06133/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2020.

## Built on by (most cited first, 15 of 15)

- [Locally Attentional SDF Diffusion for Controllable 3D Shape Generation](https://mlsanity.com/paper/2305.04461/) (2023)
- [Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches](https://mlsanity.com/paper/2104.00482/) (2021)
- [Cross-Modal 3D Shape Generation and Manipulation](https://mlsanity.com/paper/2207.11795/) (2022)
- [SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling](https://mlsanity.com/paper/2208.06880/) (2022)
- [3D VR Sketch Guided 3D Shape Prototyping and Exploration](https://mlsanity.com/paper/2306.10830/) (2023)
- [SENS: Part-Aware Sketch-based Implicit Neural Shape Modeling](https://mlsanity.com/paper/2306.06088/) (2023)
- [Fine-Grained VR Sketching: Dataset and Insights](https://mlsanity.com/paper/2209.10008/) (2022)
- [Vitruvio: 3D Building Meshes via Single Perspective Sketches](https://mlsanity.com/paper/2210.13634/) (2022)
- [SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling](https://mlsanity.com/paper/2307.00804/) (2023)
- [GroundUp: Rapid Sketch-Based 3D City Massing](https://mlsanity.com/paper/2407.12739/) (2024)
- [Interactive Sketching of Mannequin Poses](https://mlsanity.com/paper/2212.07098/) (2022)
- [Deep Sketch-Based 3D Modeling: A Survey](https://mlsanity.com/paper/2603.03287/) (2026)
- [DifferSketching: How Differently Do People Sketch 3D Objects?](https://mlsanity.com/paper/2209.08791/) (2022)
- [SingleSketch2Mesh : Generating 3D Mesh model from Sketch](https://mlsanity.com/paper/2203.03157/) (2022)
- [From Air to Wear: Personalized 3D Digital Fashion with AR/VR Immersive 3D Sketching](https://mlsanity.com/paper/2505.09998/) (2025)

## Abstract (arXiv)

Deep image-based modeling received lots of attention in recent years, yet the parallel problem of sketch-based modeling has only been briefly studied, often as a potential application. In this work, for the first time, we identify the main differences between sketch and image inputs: (i) style variance, (ii) imprecise perspective, and (iii) sparsity. We discuss why each of these differences can pose a challenge, and even make a certain class of image-based methods inapplicable. We study alternative solutions to address each of the difference. By doing so, we drive out a few important insights: (i) sparsity commonly results in an incorrect prediction of foreground versus background, (ii) diversity of human styles, if not taken into account, can lead to very poor generalization properties, and finally (iii) unless a dedicated sketching interface is used, one can not expect sketches to match a perspective of a fixed viewpoint. Finally, we compare a set of representative deep single-image modeling solutions and show how their performance can be improved to tackle sketch input by taking into consideration the identified critical differences.