# SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling

arXiv: https://arxiv.org/abs/2208.06880 · HTML: https://mlsanity.com/paper/2208.06880/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 14 Aug 2022.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019)
- [3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions](https://mlsanity.com/paper/1905.06292/) (2019)
- [Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Images](https://mlsanity.com/paper/1901.11153/) (2019)
- [Morphing and Sampling Network for Dense Point Cloud Completion](https://mlsanity.com/paper/1912.00280/) (2019)
- [MeshSDF: Differentiable Iso-Surface Extraction](https://mlsanity.com/paper/2006.03997/) (2020)
- [MarrNet: 3D Shape Reconstruction via 2.5D Sketches](https://mlsanity.com/paper/1711.03129/) (2017)
- [Pix2Vox++: Multi-scale Context-aware 3D Object Reconstruction from Single and Multiple Images](https://mlsanity.com/paper/2006.12250/) (2020)
- [Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketches](https://mlsanity.com/paper/2105.06663/) (2021)
- [Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches](https://mlsanity.com/paper/2104.00482/) (2021)
- [3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks](https://mlsanity.com/paper/1707.06375/) (2017)
- [CoReNet: Coherent 3D scene reconstruction from a single RGB image](https://mlsanity.com/paper/2004.12989/) (2020)
- [Nesti-Net: Normal Estimation for Unstructured 3D Point Clouds using Convolutional Neural Networks](https://mlsanity.com/paper/1812.00709/) (2018)
- [3D Sketching using Multi-View Deep Volumetric Prediction](https://mlsanity.com/paper/1707.08390/) (2017)
- [Pixels, voxels, and views: A study of shape representations for single view 3D object shape prediction](https://mlsanity.com/paper/1804.06032/) (2018)
- [3D Shape Reconstruction from Free-Hand Sketches](https://mlsanity.com/paper/2006.09694/) (2020)
- [Deep Sketch-Based Modeling: Tips and Tricks](https://mlsanity.com/paper/2011.06133/) (2020)
- [GraphX-Convolution for Point Cloud Deformation in 2D-to-3D Conversion](https://mlsanity.com/paper/1911.06600/) (2019)
- [Ray-ONet: Efficient 3D Reconstruction From A Single RGB Image](https://mlsanity.com/paper/2107.01899/) (2021)

## Built on by (most cited first, 14 of 14)

- [Locally Attentional SDF Diffusion for Controllable 3D Shape Generation](https://mlsanity.com/paper/2305.04461/) (2023)
- [Sketch-A-Shape: Zero-Shot Sketch-to-3D Shape Generation](https://mlsanity.com/paper/2307.03869/) (2023)
- [Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes](https://mlsanity.com/paper/2312.04043/) (2023)
- [Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings](https://mlsanity.com/paper/2411.08017/) (2024)
- [CLIPXPlore: Coupled CLIP and Shape Spaces for 3D Shape Exploration](https://mlsanity.com/paper/2306.08226/) (2023)
- [CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization](https://mlsanity.com/paper/2402.02313/) (2024)
- [S2TD-Face: Reconstruct a Detailed 3D Face with Controllable Texture from a Single Sketch](https://mlsanity.com/paper/2408.01218/) (2024)
- [Sketch3D: Style-Consistent Guidance for Sketch-to-3D Generation](https://mlsanity.com/paper/2404.01843/) (2024)
- [GroundUp: Rapid Sketch-Based 3D City Massing](https://mlsanity.com/paper/2407.12739/) (2024)
- [VRsketch2Gaussian: 3D VR Sketch Guided 3D Object Generation with Gaussian Splatting](https://mlsanity.com/paper/2503.12383/) (2025)
- [Deep Sketch-Based 3D Modeling: A Survey](https://mlsanity.com/paper/2603.03287/) (2026)
- [From Air to Wear: Personalized 3D Digital Fashion with AR/VR Immersive 3D Sketching](https://mlsanity.com/paper/2505.09998/) (2025)
- [Reconstruction of a 3D wireframe from a single line drawing via generative depth estimation](https://mlsanity.com/paper/2604.13549/) (2026)
- [CNS-Edit++: Category-Agnostic 3D Editing with Coupled Neural Shape Representation](https://mlsanity.com/paper/2607.16577/) (2026)

## Abstract (arXiv)

Reconstructing a 3D shape based on a single sketch image is challenging due to the large domain gap between a sparse, irregular sketch and a regular, dense 3D shape. Existing works try to employ the global feature extracted from sketch to directly predict the 3D coordinates, but they usually suffer from losing fine details that are not faithful to the input sketch. Through analyzing the 3D-to-2D projection process, we notice that the density map that characterizes the distribution of 2D point clouds (i.e., the probability of points projected at each location of the projection plane) can be used as a proxy to facilitate the reconstruction process. To this end, we first translate a sketch via an image translation network to a more informative 2D representation that can be used to generate a density map. Next, a 3D point cloud is reconstructed via a two-stage probabilistic sampling process: first recovering the 2D points (i.e., the x and y coordinates) by sampling the density map; and then predicting the depth (i.e., the z coordinate) by sampling the depth values at the ray determined by each 2D point. Extensive experiments are conducted, and both quantitative and qualitative results show that our proposed approach significantly outperforms other baseline methods.