mlsanityData to 7 Oct 2026

Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches

arXiv 2104.00482 ↗First posted 1 Apr 2021cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

Reconstructing 3D shape from 2D sketches has long been an open problem because the sketches only provide very sparse and ambiguous information. In this paper, we use an encoder/decoder architecture for the sketch to mesh translation. When integrated into a user interface that provides camera parameters for the sketches, this enables us to leverage its latent parametrization to represent and refine a 3D mesh so that its projections match the external contours outlined in the sketch. We will show that this approach is easy to deploy, robust to style changes, and effective. Furthermore, it can be used for shape refinement given only single pen strokes. We compare our approach to state-of-the-art methods on sketches -- both hand-drawn and synthesized -- and demonstrate that we outperform them.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 20 of its 47 references that are in the map. First posted on arXiv 1 Apr 2021.

Builds on

20 of 47 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  8. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  9. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  10. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  11. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  12. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
8 more
  1. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  2. 2020MeshSDF: Differentiable Iso-Surface Extractioncited by 112
  3. 2020FroDO: From Detections to 3D Objectscited by 51
  4. 20173D Shape Reconstruction from Sketches via Multi-view Convolutional Networkscited by 44
  5. 20173D Sketching using Multi-View Deep Volumetric Predictioncited by 36
  6. 2020Coupling Explicit and Implicit Surface Representations for Generative 3D Modelingcited by 19
  7. 2020Deep Sketch-Based Modeling: Tips and Trickscited by 15
  8. 2020UCLID-Net: Single View Reconstruction in Object Spacecited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 7; 2023: 17; 2024: 13; 2025: 7; 2026: 4.

50 papers.

Sort:

By area:

  1. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
  2. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 129
  3. 2023Control3D: Towards Controllable Text-to-3D Generationcited by 46
  4. 2023Sketch and Text Guided Diffusion Model for Colored Point Cloud Generationcited by 36
  5. 2022Text and Image Guided 3D Avatar Generation and Manipulationcited by 35
  6. 2025NANO3D: A Training-Free Approach for Efficient 3D Editing Without Maskscited by 29
  7. 2022Cross-Modal 3D Shape Generation and Manipulationcited by 27
  8. 2024Make-A-Shape: a Ten-Million-scale 3D Shape Modelcited by 25
  9. 20243Doodle: Compact Abstraction of Objects with 3D Strokescited by 21
  10. 2023Local 3D Editing via 3D Distillation of CLIP Knowledgecited by 20
  11. 2021DeepMesh: Differentiable Iso-Surface Extractioncited by 14
  12. 2022SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Samplingcited by 14
38 more
  1. 2023Sketch-A-Shape: Zero-Shot Sketch-to-3D Shape Generationcited by 14
  2. 2023Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapescited by 14
  3. 20233D VR Sketch Guided 3D Shape Prototyping and Explorationcited by 12
  4. 2024Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodingscited by 10
  5. 2023CLIPXPlore: Coupled CLIP and Shape Spaces for 3D Shape Explorationcited by 9
  6. 2023SENS: Part-Aware Sketch-based Implicit Neural Shape Modelingcited by 8
  7. 2024CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimizationcited by 8
  8. 2021HybridSDF: Combining Deep Implicit Shapes and Geometric Primitives for 3D Shape Representation and Manipulationcited by 5
  9. 2022Vitruvio: 3D Building Meshes via Single Perspective Sketchescited by 5
  10. 2023Sketch2Cloth: Sketch-based 3D Garment Generation with Unsigned Distance Fieldscited by 5
  11. 2023Deep3DSketch+: Rapid 3D Modeling from Single Free-hand Sketchescited by 5
  12. 2024Magic3DSketch: Create Colorful 3D Models From Sketch-Based 3D Modeling Guided by Text and Language-Image Pre-Trainingcited by 5
  13. 2024LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Modelcited by 5
  14. 2023Neural Implicit Shape Editing using Boundary Sensitivitycited by 4
  15. 2023SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modelingcited by 4
  16. 2024What Sketch Explainability Really Means for Downstream Taskscited by 4
  17. 2024GroundUp: Rapid Sketch-Based 3D City Massingcited by 4
  18. 2023Fine-Tuned but Zero-Shot 3D Shape Sketch View Similarity and Retrievalcited by 3
  19. 2024Toon3D: Seeing Cartoons from New Perspectivescited by 3
  20. 2024GenUDC: High Quality 3D Mesh Generation with Unsigned Dual Contouring Representationcited by 3
  21. 2024Sketch-guided Cage-based 3D Gaussian Splatting Deformationcited by 3
  22. 2025VRsketch2Gaussian: 3D VR Sketch Guided 3D Object Generation with Gaussian Splattingcited by 3
  23. 2026Deep Sketch-Based 3D Modeling: A Surveycited by 3
  24. 2022TreeSketchNet: From Sketch To 3D Tree Parameters Generationcited by 2
  25. 2022DifferSketching: How Differently Do People Sketch 3D Objects?cited by 2
  26. 2024Do Generalised Classifiers really work on Human Drawn Sketches?cited by 2
  27. 2025PASTA: Part-Aware Sketch-to-3D Shape Generation with Text-Aligned Priorcited by 2
  28. 2022Deep Learning Assisted Optimization for 3D Reconstruction from Single 2D Line Drawingscited by 1
  29. 2023Sketch2Stress: Sketching with Structural Stress Awarenesscited by 1
  30. 20233D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capabilitycited by 1
  31. 2025From Air to Wear: Personalized 3D Digital Fashion with AR/VR Immersive 3D Sketchingcited by 1
  32. 2023Strokes2Surface: Recovering Curve Networks From 4D Architectural Design Sketchescited by 0
  33. 2024Text2VP: Generative AI for Visual Programming and Parametric Modelingcited by 0
  34. 2025DiffS-NOCS: 3D Point Cloud Reconstruction through Coloring Sketches to NOCS Maps Using Diffusion Modelscited by 0
  35. 2025Order Matters: 3D Shape Generation from Sequential VR Sketchescited by 0
  36. 2026Sketch2CT: Multimodal Diffusion for Structure-Aware 3D Medical Volume Generationcited by 0
  37. 2026Reconstruction of a 3D wireframe from a single line drawing via generative depth estimationcited by 0
  38. 2026CNS-Edit++: Category-Agnostic 3D Editing with Coupled Neural Shape Representationcited by 0