mlsanityData to 14 Oct 2026

3D VR Sketch Guided 3D Shape Prototyping and Exploration

arXiv 2306.10830 ↗First posted 19 Jun 2023cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

3D shape modeling is labor-intensive, time-consuming, and requires years of expertise. To facilitate 3D shape modeling, we propose a 3D shape generation network that takes a 3D VR sketch as a condition. We assume that sketches are created by novices without art training and aim to reconstruct geometrically realistic 3D shapes of a given category. To handle potential sketch ambiguity, our method creates multiple 3D shapes that align with the original sketch's structure. We carefully design our method, training the model step-by-step and leveraging multi-modal 3D shape representation to support training with limited training data. To guarantee the realism of generated 3D shapes we leverage the normalizing flow that models the distribution of the latent space of 3D shapes. To encourage the fidelity of the generated 3D shapes to an input sketch, we propose a dedicated loss that we deploy at different stages of the training process. The code is available at https://github.com/Rowl1ng/3Dsketch2shape.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 40 of its 69 references that are in the map. First posted on arXiv 19 Jun 2023.

Builds on

40 of 69 references.

References · 40

  1. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  5. 2016Density estimation using Real NVPcited by 2,042
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  7. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  8. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  9. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  10. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  11. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
  12. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
28 more
  1. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  2. 2020SceneFormer: Indoor Scene Generation with Transformerscited by 158
  3. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  4. 2018PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attentioncited by 105
  5. 2022SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generationcited by 92
  6. 2019Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Eracited by 87
  7. 2022Towards Implicit Text-Guided 3D Shape Generationcited by 68
  8. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Modelscited by 67
  9. 2022ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Modelcited by 62
  10. 2020Multimodal Shape Completion via Conditional Generative Adversarial Networkscited by 61
  11. 20223DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Modelscited by 57
  12. 2021Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketchescited by 54
  13. 2021Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketchescited by 51
  14. 2022FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Contextcited by 49
  15. 20173D Shape Reconstruction from Sketches via Multi-view Convolutional Networkscited by 45
  16. 2020DECOR-GAN: 3D Shape Detailization by Conditional Refinementcited by 45
  17. 20173D Sketching using Multi-View Deep Volumetric Predictioncited by 37
  18. 2022Cross-Modal 3D Shape Generation and Manipulationcited by 28
  19. 2022A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketchcited by 28
  20. 2022ISS: Image as Stepping Stone for Text-Guided 3D Shape Generationcited by 27
  21. 2021ScanGAN360: A Generative Model of Realistic Scanpaths for 360circ Imagescited by 23
  22. 20203D Shape Reconstruction from Free-Hand Sketchescited by 20
  23. 2019View N-gram Network for 3D Object Retrievalcited by 18
  24. 2020Deep Sketch-Based Modeling: Tips and Trickscited by 15
  25. 2022Structure-Aware 3D VR Sketch to 3D Shape Retrievalcited by 7
  26. 2021Multimodal Shape Completion via IMLEcited by 5
  27. 2022Fine-Grained VR Sketching: Dataset and Insightscited by 5
  28. 2022Towards 3D VR-Sketch to 3D Shape Retrievalcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 6; 2026: 1.

12 papers.

  1. 2025Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modelingcited by 89
  2. 2023ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Modelcited by 39
  3. 2024It's All About Your Sketch: Democratising Sketch Control in Diffusion Modelscited by 29
  4. 2024GEM3D: GEnerative Medial Abstractions for 3D Shape Synthesiscited by 12
  5. 2024S2TD-Face: Reconstruct a Detailed 3D Face with Controllable Texture from a Single Sketchcited by 5
  6. 2025ViewCraft3D: High-Fidelity and View-Consistent 3D Vector Graphics Synthesiscited by 5
  7. 2024GroundUp: Rapid Sketch-Based 3D City Massingcited by 4
  8. 2025SketchFusion: Learning Universal Sketch Features through Fusing Foundation Modelscited by 4
  9. 2026Deep Sketch-Based 3D Modeling: A Surveycited by 3
  10. 2025UniArt: Unified 3D Representation for Generating 3D Articulated Objects with Open-Set Articulationcited by 2
  11. 2025From Air to Wear: Personalized 3D Digital Fashion with AR/VR Immersive 3D Sketchingcited by 1
  12. 2025Order Matters: 3D Shape Generation from Sequential VR Sketchescited by 0