# Sketch-based 3D Shape Retrieval using Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1504.03504 · HTML: https://mlsanity.com/paper/1504.03504/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2015.

## Built on by (most cited first, 30 of 41)

- [Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketches](https://mlsanity.com/paper/2105.06663/) (2021)
- [Sketch and Text Guided Diffusion Model for Colored Point Cloud Generation](https://mlsanity.com/paper/2308.02874/) (2023)
- [Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis](https://mlsanity.com/paper/2104.05703/) (2021)
- [ScribbleLight: Single Image Indoor Relighting with Scribbles](https://mlsanity.com/paper/2411.17696/) (2024)
- [AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints](https://mlsanity.com/paper/2205.10636/) (2022)
- [An Efficient Framework for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2102.04016/) (2021)
- [Adaptive Fine-Grained Sketch-Based Image Retrieval](https://mlsanity.com/paper/2207.01723/) (2022)
- [Domain Disentangled Generative Adversarial Network for Zero-Shot Sketch-Based 3D Shape Retrieval](https://mlsanity.com/paper/2202.11948/) (2022)
- [CrossATNet - A Novel Cross-Attention Based Framework for Sketch-Based Image Retrieval](https://mlsanity.com/paper/2104.09918/) (2021)
- [Zero-Shot Sketch Based Image Retrieval using Graph Transformer](https://mlsanity.com/paper/2201.10185/) (2022)
- [Structure-Aware 3D VR Sketch to 3D Shape Retrieval](https://mlsanity.com/paper/2209.09043/) (2022)
- [Conjugate Product Graphs for Globally Optimal 2D-3D Shape Matching](https://mlsanity.com/paper/2211.11589/) (2022)
- [Not with my name! Inferring artists' names of input strings employed by Diffusion Models](https://mlsanity.com/paper/2307.13527/) (2023)
- ['CADSketchNet' -- An Annotated Sketch dataset for 3D CAD Model Retrieval with Deep Neural Networks](https://mlsanity.com/paper/2107.06212/) (2021)
- [Fine-Grained VR Sketching: Dataset and Insights](https://mlsanity.com/paper/2209.10008/) (2022)
- [Sketch2Cloth: Sketch-based 3D Garment Generation with Unsigned Distance Fields](https://mlsanity.com/paper/2303.00167/) (2023)
- [Deep3DSketch+: Rapid 3D Modeling from Single Free-hand Sketches](https://mlsanity.com/paper/2309.13006/) (2023)
- [Magic3DSketch: Create Colorful 3D Models From Sketch-Based 3D Modeling Guided by Text and Language-Image Pre-Training](https://mlsanity.com/paper/2407.19225/) (2024)
- [Edge Augmentation for Large-Scale Sketch Recognition without Sketches](https://mlsanity.com/paper/2202.13164/) (2022)
- [Towards 3D VR-Sketch to 3D Shape Retrieval](https://mlsanity.com/paper/2209.10020/) (2022)
- [SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling](https://mlsanity.com/paper/2307.00804/) (2023)
- [Scene Designer: a Unified Model for Scene Search and Synthesis from Sketch](https://mlsanity.com/paper/2108.07353/) (2021)
- [Localizing Infinity-shaped fishes: Sketch-guided object localization in the wild](https://mlsanity.com/paper/2109.11874/) (2021)
- [Query-guided Attention in Vision Transformers for Localizing Objects Using a Single Sketch](https://mlsanity.com/paper/2303.08784/) (2023)
- [Fine-Tuned but Zero-Shot 3D Shape Sketch View Similarity and Retrieval](https://mlsanity.com/paper/2306.08541/) (2023)
- [Deep Sketch-Based 3D Modeling: A Survey](https://mlsanity.com/paper/2603.03287/) (2026)
- [Sketch-an-Anchor: Sub-epoch Fast Model Adaptation for Zero-shot Sketch-based Image Retrieval](https://mlsanity.com/paper/2303.16769/) (2023)
- [Active Learning for Fine-Grained Sketch-Based Image Retrieval](https://mlsanity.com/paper/2309.08743/) (2023)
- [Sketch Input Method Editor: A Comprehensive Dataset and Methodology for Systematic Input Recognition](https://mlsanity.com/paper/2311.18254/) (2023)
- [U-Sketch: An Efficient Approach for Sketch to Image Diffusion Models](https://mlsanity.com/paper/2403.18425/) (2024)

## Abstract (arXiv)

Retrieving 3D models from 2D human sketches has received considerable attention in the areas of graphics, image retrieval, and computer vision. Almost always in state of the art approaches a large amount of "best views" are computed for 3D models, with the hope that the query sketch matches one of these 2D projections of 3D models using predefined features. We argue that this two stage approach (view selection -- matching) is pragmatic but also problematic because the "best views" are subjective and ambiguous, which makes the matching inputs obscure. This imprecise nature of matching further makes it challenging to choose features manually. Instead of relying on the elusive concept of "best views" and the hand-crafted features, we propose to define our views using a minimalism approach and learn features for both sketches and views. Specifically, we drastically reduce the number of views to only two predefined directions for the whole dataset. Then, we learn two Siamese Convolutional Neural Networks (CNNs), one for the views and one for the sketches. The loss function is defined on the within-domain as well as the cross-domain similarities. Our experiments on three benchmark datasets demonstrate that our method is significantly better than state of the art approaches, and outperforms them in all conventional metrics.