# Zero-Shot Sketch Based Image Retrieval using Graph Transformer

arXiv: https://arxiv.org/abs/2201.10185 · HTML: https://mlsanity.com/paper/2201.10185/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 25 Jan 2022.

## Builds on (most cited first, 25 of 25)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [Feature Generating Networks for Zero-Shot Learning](https://mlsanity.com/paper/1712.00981/) (2017)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [Beyond triplet loss: a deep quadruplet network for person re-identification](https://mlsanity.com/paper/1704.01719/) (2017)
- [Semantic Autoencoder for Zero-Shot Learning](https://mlsanity.com/paper/1704.08345/) (2017)
- [Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/1904.03451/) (2019)
- [Zero-Shot Learning via Semantic Similarity Embedding](https://mlsanity.com/paper/1509.04767/) (2015)
- [A Zero-Shot Framework for Sketch-based Image Retrieval](https://mlsanity.com/paper/1807.11724/) (2018)
- [Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval](https://mlsanity.com/paper/1703.05605/) (2017)
- [Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/1903.03372/) (2019)
- [Zero-Shot Sketch-Image Hashing](https://mlsanity.com/paper/1803.02284/) (2018)
- [Sketch-based 3D Shape Retrieval using Convolutional Neural Networks](https://mlsanity.com/paper/1504.03504/) (2015)
- [Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/1904.03208/) (2019)
- [Zero-Shot Learning via Joint Latent Similarity Embedding](https://mlsanity.com/paper/1511.04512/) (2015)
- [An Efficient Framework for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2102.04016/) (2021)
- [Stacked Adversarial Network for Zero-Shot Sketch based Image Retrieval](https://mlsanity.com/paper/2001.06657/) (2020)
- [Semantically Tied Paired Cycle Consistency for Any-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/2006.11397/) (2020)
- [CrossATNet - A Novel Cross-Attention Based Framework for Sketch-Based Image Retrieval](https://mlsanity.com/paper/2104.09918/) (2021)
- [BDA-SketRet: Bi-Level Domain Adaptation for Zero-Shot SBIR](https://mlsanity.com/paper/2201.06570/) (2022)
- [Generative Model for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/1904.08542/) (2019)

## Built on by (most cited first, 7 of 7)

- [Elevating All Zero-Shot Sketch-Based Image Retrieval Through Multimodal Prompt Learning](https://mlsanity.com/paper/2407.04207/) (2024)
- [Transformers and CNNs both Beat Humans on SBIR](https://mlsanity.com/paper/2209.06629/) (2022)
- [If At First You Don't Succeed: Test Time Re-ranking for Zero-shot, Cross-domain Retrieval](https://mlsanity.com/paper/2303.17703/) (2023)
- [Cross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrieval](https://mlsanity.com/paper/2407.00979/) (2024)
- [SketchTriplet: Self-Supervised Scenarized Sketch-Text-Image Triplet Generation](https://mlsanity.com/paper/2405.18801/) (2024)
- [Dynamic Multi-level Weighted Alignment Network for Zero-shot Sketch-based Image Retrieval](https://mlsanity.com/paper/2511.00925/) (2025)
- [SeCo-SBIR: Semantically Consistent Prompt Learning for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2608.03120/) (2026)

## Abstract (arXiv)

The performance of a zero-shot sketch-based image retrieval (ZS-SBIR) task is primarily affected by two challenges. The substantial domain gap between image and sketch features needs to be bridged, while at the same time the side information has to be chosen tactfully. Existing literature has shown that varying the semantic side information greatly affects the performance of ZS-SBIR. To this end, we propose a novel graph transformer based zero-shot sketch-based image retrieval (GTZSR) framework for solving ZS-SBIR tasks which uses a novel graph transformer to preserve the topology of the classes in the semantic space and propagates the context-graph of the classes within the embedding features of the visual space. To bridge the domain gap between the visual features, we propose minimizing the Wasserstein distance between images and sketches in a learned domain-shared space. We also propose a novel compatibility loss that further aligns the two visual domains by bridging the domain gap of one class with respect to the domain gap of all other classes in the training set. Experimental results obtained on the extended Sketchy, TU-Berlin, and QuickDraw datasets exhibit sharp improvements over the existing state-of-the-art methods in both ZS-SBIR and generalized ZS-SBIR.