# Symmetrical Bidirectional Knowledge Alignment for Zero-Shot Sketch-Based Image Retrieval

arXiv: https://arxiv.org/abs/2312.10320 · HTML: https://mlsanity.com/paper/2312.10320/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 52 references that are in the map.
- First posted on arXiv 16 Dec 2023.

## Builds on (most cited first, 26 of 26)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Teacher-Student Curriculum Learning](https://mlsanity.com/paper/1707.00183/) (2017)
- [Multi-Task Self-Training for Learning General Representations](https://mlsanity.com/paper/2108.11353/) (2021)
- [Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/1904.03451/) (2019)
- [CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Not](https://mlsanity.com/paper/2303.13440/) (2023)
- [A Zero-Shot Framework for Sketch-based Image Retrieval](https://mlsanity.com/paper/1807.11724/) (2018)
- [Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/1903.03372/) (2019)
- [Zero-Shot Sketch-Image Hashing](https://mlsanity.com/paper/1803.02284/) (2018)
- [Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/1904.03208/) (2019)
- [Sketch3T: Test-Time Training for Zero-Shot SBIR](https://mlsanity.com/paper/2203.14691/) (2022)
- [Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval](https://mlsanity.com/paper/2203.14817/) (2022)
- [Towards Defending against Adversarial Examples via Attack-Invariant Features](https://mlsanity.com/paper/2106.05036/) (2021)
- [Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style](https://mlsanity.com/paper/2303.14348/) (2023)
- [Partially Does It: Towards Scene-Level FG-SBIR with Partial Input](https://mlsanity.com/paper/2203.14804/) (2022)
- [An Efficient Framework for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2102.04016/) (2021)
- [Domain-Smoothing Network for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2106.11841/) (2021)
- [FedForgery: Generalized Face Forgery Detection with Residual Federated Learning](https://mlsanity.com/paper/2210.09563/) (2022)
- [Semi-Heterogeneous Three-Way Joint Embedding Network for Sketch-Based Image Retrieval](https://mlsanity.com/paper/1911.04470/) (2019)
- [Data-Free Sketch-Based Image Retrieval](https://mlsanity.com/paper/2303.07775/) (2023)
- [Deep Semantic Parsing of Freehand Sketches with Homogeneous Transformation, Soft-Weighted Loss, and Staged Learning](https://mlsanity.com/paper/1910.06023/) (2019)
- [ACNet: Approaching-and-Centralizing Network for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2111.12757/) (2021)
- [Adapt and Align to Improve Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2305.05144/) (2023)
- [Distribution Aligned Feature Clustering for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2301.06685/) (2023)

## Abstract (arXiv)

This paper studies the problem of zero-shot sketch-based image retrieval (ZS-SBIR), which aims to use sketches from unseen categories as queries to match the images of the same category. Due to the large cross-modality discrepancy, ZS-SBIR is still a challenging task and mimics realistic zero-shot scenarios. The key is to leverage transferable knowledge from the pre-trained model to improve generalizability. Existing researchers often utilize the simple fine-tuning training strategy or knowledge distillation from a teacher model with fixed parameters, lacking efficient bidirectional knowledge alignment between student and teacher models simultaneously for better generalization. In this paper, we propose a novel Symmetrical Bidirectional Knowledge Alignment for zero-shot sketch-based image retrieval (SBKA). The symmetrical bidirectional knowledge alignment learning framework is designed to effectively learn mutual rich discriminative information between teacher and student models to achieve the goal of knowledge alignment. Instead of the former one-to-one cross-modality matching in the testing stage, a one-to-many cluster cross-modality matching method is proposed to leverage the inherent relationship of intra-class images to reduce the adverse effects of the existing modality gap. Experiments on several representative ZS-SBIR datasets (Sketchy Ext dataset, TU-Berlin Ext dataset and QuickDraw Ext dataset) prove the proposed algorithm can achieve superior performance compared with state-of-the-art methods.