# Active Learning for Fine-Grained Sketch-Based Image Retrieval

arXiv: https://arxiv.org/abs/2309.08743 · HTML: https://mlsanity.com/paper/2309.08743/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 49 references that are in the map.
- First posted on arXiv 15 Sep 2023.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Audio-Visual Instance Discrimination with Cross-Modal Agreement](https://mlsanity.com/paper/2004.12943/) (2020)
- [A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis](https://mlsanity.com/paper/1910.02923/) (2019)
- [ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation](https://mlsanity.com/paper/1911.11789/) (2019)
- [Diverse mini-batch Active Learning](https://mlsanity.com/paper/1901.05954/) (2019)
- [Task-Aware Variational Adversarial Active Learning](https://mlsanity.com/paper/2002.04709/) (2020)
- [CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Not](https://mlsanity.com/paper/2303.13440/) (2023)
- [State-Relabeling Adversarial Active Learning](https://mlsanity.com/paper/2004.04943/) (2020)
- [Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval](https://mlsanity.com/paper/2002.10310/) (2020)
- [StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval](https://mlsanity.com/paper/2103.15706/) (2021)
- [Learning to Sketch with Shortcut Cycle Consistency](https://mlsanity.com/paper/1805.00247/) (2018)
- [Label Efficient Semi-Supervised Learning via Graph Filtering](https://mlsanity.com/paper/1901.09993/) (2019)
- [Sketch-based 3D Shape Retrieval using Convolutional Neural Networks](https://mlsanity.com/paper/1504.03504/) (2015)
- [More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval](https://mlsanity.com/paper/2103.13990/) (2021)
- [Learning Deep Sketch Abstraction](https://mlsanity.com/paper/1804.04804/) (2018)
- [Sketch3T: Test-Time Training for Zero-Shot SBIR](https://mlsanity.com/paper/2203.14691/) (2022)
- [Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval](https://mlsanity.com/paper/2007.15103/) (2020)
- [DEAL: Difficulty-aware Active Learning for Semantic Segmentation](https://mlsanity.com/paper/2010.08705/) (2020)
- [VaB-AL: Incorporating Class Imbalance and Difficulty with Variational Bayes for Active Learning](https://mlsanity.com/paper/2003.11249/) (2020)

## Built on by (most cited first, 2 of 2)

- [How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?](https://mlsanity.com/paper/2403.07203/) (2024)
- [Pseudo-triplet Guided Few-shot Composed Image Retrieval](https://mlsanity.com/paper/2407.06001/) (2024)

## Abstract (arXiv)

The ability to retrieve a photo by mere free-hand sketching highlights the immense potential of Fine-grained sketch-based image retrieval (FG-SBIR). However, its rapid practical adoption, as well as scalability, is limited by the expense of acquiring faithful sketches for easily available photo counterparts. A solution to this problem is Active Learning, which could minimise the need for labeled sketches while maximising performance. Despite extensive studies in the field, there exists no work that utilises it for reducing sketching effort in FG-SBIR tasks. To this end, we propose a novel active learning sampling technique that drastically minimises the need for drawing photo sketches. Our proposed approach tackles the trade-off between uncertainty and diversity by utilising the relationship between the existing photo-sketch pair to a photo that does not have its sketch and augmenting this relation with its intermediate representations. Since our approach relies only on the underlying data distribution, it is agnostic of the modelling approach and hence is applicable to other cross-modal instance-level retrieval tasks as well. With experimentation over two publicly available fine-grained SBIR datasets ChairV2 and ShoeV2, we validate our approach and reveal its superiority over adapted baselines.