# 3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks

arXiv: https://arxiv.org/abs/1707.06375 · HTML: https://mlsanity.com/paper/1707.06375/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2017.

## Built on by (most cited first, 30 of 45)

- [Locally Attentional SDF Diffusion for Controllable 3D Shape Generation](https://mlsanity.com/paper/2305.04461/) (2023)
- [SKED: Sketch-guided Text-based 3D Editing](https://mlsanity.com/paper/2303.10735/) (2023)
- [Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketches](https://mlsanity.com/paper/2105.06663/) (2021)
- [Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches](https://mlsanity.com/paper/2104.00482/) (2021)
- [ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators](https://mlsanity.com/paper/2108.09432/) (2021)
- [Make-A-Shape: a Ten-Million-scale 3D Shape Model](https://mlsanity.com/paper/2401.11067/) (2024)
- [Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis](https://mlsanity.com/paper/2104.05703/) (2021)
- [3D-aware Conditional Image Synthesis](https://mlsanity.com/paper/2302.08509/) (2023)
- [Augmenting Implicit Neural Shape Representations with Explicit Deformation Fields](https://mlsanity.com/paper/2108.08931/) (2021)
- [SketchLattice: Latticed Representation for Sketch Manipulation](https://mlsanity.com/paper/2108.11636/) (2021)
- [PAniC-3D: Stylized Single-view 3D Reconstruction from Portraits of Anime Characters](https://mlsanity.com/paper/2303.14587/) (2023)
- [SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling](https://mlsanity.com/paper/2208.06880/) (2022)
- [Sketch-A-Shape: Zero-Shot Sketch-to-3D Shape Generation](https://mlsanity.com/paper/2307.03869/) (2023)
- [T2TD: Text-3D Generation Model based on Prior Knowledge Guidance](https://mlsanity.com/paper/2305.15753/) (2023)
- [3D VR Sketch Guided 3D Shape Prototyping and Exploration](https://mlsanity.com/paper/2306.10830/) (2023)
- [Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings](https://mlsanity.com/paper/2411.08017/) (2024)
- [SimpModeling: Sketching Implicit Field to Guide Mesh Modeling for 3D Animalmorphic Head Design](https://mlsanity.com/paper/2108.02548/) (2021)
- [What's the Situation with Intelligent Mesh Generation: A Survey and Perspectives](https://mlsanity.com/paper/2211.06009/) (2022)
- [External Knowledge Enhanced 3D Scene Generation from Sketch](https://mlsanity.com/paper/2403.14121/) (2024)
- [SENS: Part-Aware Sketch-based Implicit Neural Shape Modeling](https://mlsanity.com/paper/2306.06088/) (2023)
- [Sketch2PQ: Freeform Planar Quadrilateral Mesh Design via a Single Sketch](https://mlsanity.com/paper/2201.09367/) (2022)
- [Sketch2Cloth: Sketch-based 3D Garment Generation with Unsigned Distance Fields](https://mlsanity.com/paper/2303.00167/) (2023)
- [SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling](https://mlsanity.com/paper/2307.00804/) (2023)
- [Sketch3D: Style-Consistent Guidance for Sketch-to-3D Generation](https://mlsanity.com/paper/2404.01843/) (2024)
- [GroundUp: Rapid Sketch-Based 3D City Massing](https://mlsanity.com/paper/2407.12739/) (2024)
- [Sketch-based Normal Map Generation with Geometric Sampling](https://mlsanity.com/paper/2104.11554/) (2021)
- [Interactive Sketching of Mannequin Poses](https://mlsanity.com/paper/2212.07098/) (2022)
- [Query-guided Attention in Vision Transformers for Localizing Objects Using a Single Sketch](https://mlsanity.com/paper/2303.08784/) (2023)
- [Fine-Tuned but Zero-Shot 3D Shape Sketch View Similarity and Retrieval](https://mlsanity.com/paper/2306.08541/) (2023)
- [PASTA: Part-Aware Sketch-to-3D Shape Generation with Text-Aligned Prior](https://mlsanity.com/paper/2503.12834/) (2025)

## Abstract (arXiv)

We propose a method for reconstructing 3D shapes from 2D sketches in the form of line drawings. Our method takes as input a single sketch, or multiple sketches, and outputs a dense point cloud representing a 3D reconstruction of the input sketch(es). The point cloud is then converted into a polygon mesh. At the heart of our method lies a deep, encoder-decoder network. The encoder converts the sketch into a compact representation encoding shape information. The decoder converts this representation into depth and normal maps capturing the underlying surface from several output viewpoints. The multi-view maps are then consolidated into a 3D point cloud by solving an optimization problem that fuses depth and normals across all viewpoints. Based on our experiments, compared to other methods, such as volumetric networks, our architecture offers several advantages, including more faithful reconstruction, higher output surface resolution, better preservation of topology and shape structure.