# Semantically Tied Paired Cycle Consistency for Any-Shot Sketch-based Image Retrieval

arXiv: https://arxiv.org/abs/2006.11397 · HTML: https://mlsanity.com/paper/2006.11397/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2020.

## Built on by (most cited first, 10 of 10)

- [TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning](https://mlsanity.com/paper/2112.08643/) (2021)
- [Adaptive Fine-Grained Sketch-Based Image Retrieval](https://mlsanity.com/paper/2207.01723/) (2022)
- [BDA-SketRet: Bi-Level Domain Adaptation for Zero-Shot SBIR](https://mlsanity.com/paper/2201.06570/) (2022)
- [Zero-Shot Sketch Based Image Retrieval using Graph Transformer](https://mlsanity.com/paper/2201.10185/) (2022)
- [Data-Free Sketch-Based Image Retrieval](https://mlsanity.com/paper/2303.07775/) (2023)
- [EGANS: Evolutionary Generative Adversarial Network Search for Zero-Shot Learning](https://mlsanity.com/paper/2308.09915/) (2023)
- [Implicit and Explicit Attention for Zero-Shot Learning](https://mlsanity.com/paper/2110.00860/) (2021)
- [Fine-Grained VR Sketching: Dataset and Insights](https://mlsanity.com/paper/2209.10008/) (2022)
- [Cross-Modal Fusion Distillation for Fine-Grained Sketch-Based Image Retrieval](https://mlsanity.com/paper/2210.10486/) (2022)
- [Sketch-an-Anchor: Sub-epoch Fast Model Adaptation for Zero-shot Sketch-based Image Retrieval](https://mlsanity.com/paper/2303.16769/) (2023)

## Abstract (arXiv)

Low-shot sketch-based image retrieval is an emerging task in computer vision, allowing to retrieve natural images relevant to hand-drawn sketch queries that are rarely seen during the training phase. Related prior works either require aligned sketch-image pairs that are costly to obtain or inefficient memory fusion layer for mapping the visual information to a semantic space. In this paper, we address any-shot, i.e. zero-shot and few-shot, sketch-based image retrieval (SBIR) tasks, where we introduce the few-shot setting for SBIR. For solving these tasks, we propose a semantically aligned paired cycle-consistent generative adversarial network (SEM-PCYC) for any-shot SBIR, where each branch of the generative adversarial network maps the visual information from sketch and image to a common semantic space via adversarial training. Each of these branches maintains cycle consistency that only requires supervision at the category level, and avoids the need of aligned sketch-image pairs. A classification criteria on the generators' outputs ensures the visual to semantic space mapping to be class-specific. Furthermore, we propose to combine textual and hierarchical side information via an auto-encoder that selects discriminating side information within a same end-to-end model. Our results demonstrate a significant boost in any-shot SBIR performance over the state-of-the-art on the extended version of the challenging Sketchy, TU-Berlin and QuickDraw datasets.