# Relation-Aware Meta-Learning for Zero-shot Sketch-Based Image Retrieval

arXiv: https://arxiv.org/abs/2412.00120 · HTML: https://mlsanity.com/paper/2412.00120/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 28 Nov 2024.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Semantic Autoencoder for Zero-Shot Learning](https://mlsanity.com/paper/1704.08345/) (2017)
- [Deep Metric Learning with Angular Loss](https://mlsanity.com/paper/1708.01682/) (2017)
- [Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/1904.03451/) (2019)
- [A Zero-Shot Framework for Sketch-based Image Retrieval](https://mlsanity.com/paper/1807.11724/) (2018)
- [Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval](https://mlsanity.com/paper/1703.05605/) (2017)
- [Learning Dynamic Memory Networks for Object Tracking](https://mlsanity.com/paper/1803.07268/) (2018)
- [Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/1903.03372/) (2019)
- [Zero-Shot Sketch-Image Hashing](https://mlsanity.com/paper/1803.02284/) (2018)
- [Learning to Adapt Invariance in Memory for Person Re-identification](https://mlsanity.com/paper/1908.00485/) (2019)
- [Sketch3T: Test-Time Training for Zero-Shot SBIR](https://mlsanity.com/paper/2203.14691/) (2022)
- [Learning to Learn: Meta-Critic Networks for Sample Efficient Learning](https://mlsanity.com/paper/1706.09529/) (2017)
- [Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style](https://mlsanity.com/paper/2303.14348/) (2023)
- [Variational Interaction Information Maximization for Cross-domain Disentanglement](https://mlsanity.com/paper/2012.04251/) (2020)
- [Domain-Smoothing Network for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2106.11841/) (2021)
- [MetaKG: Meta-learning on Knowledge Graph for Cold-start Recommendation](https://mlsanity.com/paper/2202.03851/) (2022)
- [Many-Class Few-Shot Learning on Multi-Granularity Class Hierarchy](https://mlsanity.com/paper/2006.15479/) (2020)
- [Part-Whole Relational Fusion Towards Multi-Modal Scene Understanding](https://mlsanity.com/paper/2410.14944/) (2024)
- [Intra-class Adaptive Augmentation with Neighbor Correction for Deep Metric Learning](https://mlsanity.com/paper/2211.16264/) (2022)
- [Cross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrieval](https://mlsanity.com/paper/2407.00979/) (2024)

## Built on by (most cited first, 1 of 1)

- [Thinking Before Retrieving: Robust Zero-Shot Composed Image Retrieval via Strategic Planning and Self-Criticism](https://mlsanity.com/paper/2606.31222/) (2026)

## Abstract (arXiv)

Sketch-based image retrieval (SBIR) relies on free-hand sketches to retrieve natural photos within the same class. However, its practical application is limited by its inability to retrieve classes absent from the training set. To address this limitation, the task has evolved into Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR), where model performance is evaluated on unseen categories. Traditional SBIR primarily focuses on narrowing the domain gap between photo and sketch modalities. However, in the zero-shot setting, the model not only needs to address this cross-modal discrepancy but also requires a strong generalization capability to transfer knowledge to unseen categories. To this end, we propose a novel framework for ZS-SBIR that employs a pair-based relation-aware quadruplet loss to bridge feature gaps. By incorporating two negative samples from different modalities, the approach prevents positive features from becoming disproportionately distant from one modality while remaining close to another, thus enhancing inter-class separability. We also propose a Relation-Aware Meta-Learning Network (RAMLN) to obtain the margin, a hyper-parameter of cross-modal quadruplet loss, to improve the generalization ability of the model. RAMLN leverages external memory to store feature information, which it utilizes to assign optimal margin values. Experimental results obtained on the extended Sketchy and TU-Berlin datasets show a sharp improvement over existing state-of-the-art methods in ZS-SBIR.