# An Efficient Framework for Zero-Shot Sketch-Based Image Retrieval

arXiv: https://arxiv.org/abs/2102.04016 · HTML: https://mlsanity.com/paper/2102.04016/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 48 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Beyond triplet loss: a deep quadruplet network for person re-identification](https://mlsanity.com/paper/1704.01719/) (2017)
- [Deep Image Retrieval: Learning global representations for image search](https://mlsanity.com/paper/1604.01325/) (2016)
- [Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/1904.03451/) (2019)
- [A Zero-Shot Framework for Sketch-based Image Retrieval](https://mlsanity.com/paper/1807.11724/) (2018)
- [Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval](https://mlsanity.com/paper/1703.05605/) (2017)
- [Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/1903.03372/) (2019)
- [Zero-Shot Sketch-Image Hashing](https://mlsanity.com/paper/1803.02284/) (2018)
- [Sketch-based 3D Shape Retrieval using Convolutional Neural Networks](https://mlsanity.com/paper/1504.03504/) (2015)
- [Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/1904.03208/) (2019)
- [Recent Advances in Zero-shot Recognition](https://mlsanity.com/paper/1710.04837/) (2017)
- [Stacked Adversarial Network for Zero-Shot Sketch based Image Retrieval](https://mlsanity.com/paper/2001.06657/) (2020)
- [Semi-Heterogeneous Three-Way Joint Embedding Network for Sketch-Based Image Retrieval](https://mlsanity.com/paper/1911.04470/) (2019)
- [Component-based Attention for Large-scale Trademark Retrieval](https://mlsanity.com/paper/1811.02746/) (2018)
- [Generative Model for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/1904.08542/) (2019)

## Built on by (most cited first, 15 of 15)

- [A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketch](https://mlsanity.com/paper/2208.03354/) (2022)
- [Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style](https://mlsanity.com/paper/2303.14348/) (2023)
- [BDA-SketRet: Bi-Level Domain Adaptation for Zero-Shot SBIR](https://mlsanity.com/paper/2201.06570/) (2022)
- [Zero-Shot Sketch Based Image Retrieval using Graph Transformer](https://mlsanity.com/paper/2201.10185/) (2022)
- [Zero-Shot Multi-Modal Artist-Controlled Retrieval and Exploration of 3D Object Sets](https://mlsanity.com/paper/2209.00682/) (2022)
- [ACNet: Approaching-and-Centralizing Network for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2111.12757/) (2021)
- [Adapt and Align to Improve Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2305.05144/) (2023)
- [Three-Stream Joint Network for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2204.05666/) (2022)
- [LANDMARK: Language-guided Representation Enhancement Framework for Scene Graph Generation](https://mlsanity.com/paper/2303.01080/) (2023)
- [Sketch-an-Anchor: Sub-epoch Fast Model Adaptation for Zero-shot Sketch-based Image Retrieval](https://mlsanity.com/paper/2303.16769/) (2023)
- [If At First You Don't Succeed: Test Time Re-ranking for Zero-shot, Cross-domain Retrieval](https://mlsanity.com/paper/2303.17703/) (2023)
- [Cross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrieval](https://mlsanity.com/paper/2407.00979/) (2024)
- [SketchTriplet: Self-Supervised Scenarized Sketch-Text-Image Triplet Generation](https://mlsanity.com/paper/2405.18801/) (2024)
- [On Training Sketch Recognizers for New Domains](https://mlsanity.com/paper/2104.08850/) (2021)
- [Symmetrical Bidirectional Knowledge Alignment for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2312.10320/) (2023)

## Abstract (arXiv)

Recently, Zero-shot Sketch-based Image Retrieval (ZS-SBIR) has attracted the attention of the computer vision community due to it's real-world applications, and the more realistic and challenging setting than found in SBIR. ZS-SBIR inherits the main challenges of multiple computer vision problems including content-based Image Retrieval (CBIR), zero-shot learning and domain adaptation. The majority of previous studies using deep neural networks have achieved improved results through either projecting sketch and images into a common low-dimensional space or transferring knowledge from seen to unseen classes. However, those approaches are trained with complex frameworks composed of multiple deep convolutional neural networks (CNNs) and are dependent on category-level word labels. This increases the requirements on training resources and datasets. In comparison, we propose a simple and efficient framework that does not require high computational training resources, and can be trained on datasets without semantic categorical labels. Furthermore, at training and inference stages our method only uses a single CNN. In this work, a pre-trained ImageNet CNN (e.g., ResNet50) is fine-tuned with three proposed learning objects: domain-aware quadruplet loss, semantic classification loss, and semantic knowledge preservation loss. The domain-aware quadruplet and semantic classification losses are introduced to learn discriminative, semantic and domain invariant features through considering ZS-SBIR as object detection and verification problem. ...