# Deep Learning for Instance Retrieval: A Survey

arXiv: https://arxiv.org/abs/2101.11282 · HTML: https://mlsanity.com/paper/2101.11282/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Builds on 80 of its 171 references that are in the map.
- First posted on arXiv 27 Jan 2021.

## Builds on (most cited first, 30 of 80)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks](https://mlsanity.com/paper/1312.6229/) (2013)
- [A Survey of the Recent Architectures of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1901.06032/) (2019)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking](https://mlsanity.com/paper/1803.11285/) (2018)
- [Learning Fine-grained Image Similarity with Deep Ranking](https://mlsanity.com/paper/1404.4661/) (2014)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [Particular object retrieval with integral max-pooling of CNN activations](https://mlsanity.com/paper/1511.05879/) (2015)
- [Deep Image Retrieval: Learning global representations for image search](https://mlsanity.com/paper/1604.01325/) (2016)
- [Return of the Devil in the Details: Delving Deep into Convolutional Nets](https://mlsanity.com/paper/1405.3531/) (2014)
- [End-to-end Learning of Deep Visual Representations for Image Retrieval](https://mlsanity.com/paper/1610.07940/) (2016)
- [A Survey on Learning to Hash](https://mlsanity.com/paper/1606.00185/) (2016)
- [A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses](https://mlsanity.com/paper/2003.08983/) (2020)
- [Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval](https://mlsanity.com/paper/2007.12163/) (2020)

## Built on by (most cited first, 30 of 70)

- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [MixVPR: Feature Mixing for Visual Place Recognition](https://mlsanity.com/paper/2303.02190/) (2023)
- [MatFormer: Nested Transformer for Elastic Inference](https://mlsanity.com/paper/2310.07707/) (2023)
- [Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting](https://mlsanity.com/paper/2407.08223/) (2024)
- [Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks](https://mlsanity.com/paper/2310.19909/) (2023)
- [Global Features are All You Need for Image Retrieval and Reranking](https://mlsanity.com/paper/2308.06954/) (2023)
- [Handcrafted Histological Transformer (H2T): Unsupervised Representation of Whole Slide Images](https://mlsanity.com/paper/2202.07001/) (2022)
- [Unleashing Unlabeled Data: A Paradigm for Cross-View Geo-Localization](https://mlsanity.com/paper/2403.14198/) (2024)
- [POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Reference](https://mlsanity.com/paper/2305.15727/) (2023)
- [Towards Universal Image Embeddings: A Large-Scale Dataset and Challenge for Generic Image Representations](https://mlsanity.com/paper/2309.01858/) (2023)
- [Close, But Not There: Boosting Geographic Distance Sensitivity in Visual Place Recognition](https://mlsanity.com/paper/2407.02422/) (2024)
- [Target before Shooting: Accurate Anomaly Detection and Localization under One Millisecond via Cascade Patch Retrieval](https://mlsanity.com/paper/2308.06748/) (2023)
- [Controllable Textual Inversion for Personalized Text-to-Image Generation](https://mlsanity.com/paper/2304.05265/) (2023)
- [Uni-Retrieval: A Multi-Style Retrieval Framework for STEM's Education](https://mlsanity.com/paper/2502.05863/) (2025)
- [SAAN: Similarity-aware attention flow network for change detection with VHR remote sensing images](https://mlsanity.com/paper/2308.14570/) (2023)
- [Have We Ever Encountered This Before? Retrieving Out-of-Distribution Road Obstacles from Driving Scenes](https://mlsanity.com/paper/2309.04302/) (2023)
- [VoxDet: Voxel Learning for Novel Instance Detection](https://mlsanity.com/paper/2305.17220/) (2023)
- [Let All be Whitened: Multi-teacher Distillation for Efficient Visual Retrieval](https://mlsanity.com/paper/2312.09716/) (2023)
- [Analysis and Validation of Image Search Engines in Histopathology](https://mlsanity.com/paper/2401.03271/) (2024)
- [LInK: Learning Joint Representations of Design and Performance Spaces through Contrastive Learning for Mechanism Synthesis](https://mlsanity.com/paper/2405.20592/) (2024)
- [Zero-shot Composed Image Retrieval Considering Query-target Relationship Leveraging Masked Image-text Pairs](https://mlsanity.com/paper/2406.18836/) (2024)
- [MatMamba: A Matryoshka State Space Model](https://mlsanity.com/paper/2410.06718/) (2024)
- [ConText-CIR: Learning from Concepts in Text for Composed Image Retrieval](https://mlsanity.com/paper/2505.20764/) (2025)
- [Region-based Contrastive Pretraining for Medical Image Retrieval with Anatomic Query](https://mlsanity.com/paper/2305.05598/) (2023)
- [Learning Unknowns from Unknowns: Diversified Negative Prototypes Generator for Few-Shot Open-Set Recognition](https://mlsanity.com/paper/2408.13373/) (2024)
- [PanoVPR: Towards Unified Perspective-to-Equirectangular Visual Place Recognition via Sliding Windows across the Panoramic View](https://mlsanity.com/paper/2303.14095/) (2023)
- [A Flexible Nadaraya-Watson Head Can Offer Explainable and Calibrated Classification](https://mlsanity.com/paper/2212.03411/) (2022)
- [Leveraging BEV Representation for 360-degree Visual Place Recognition](https://mlsanity.com/paper/2305.13814/) (2023)
- [Evaluating Pre-trained Convolutional Neural Networks and Foundation Models as Feature Extractors for Content-based Medical Image Retrieval](https://mlsanity.com/paper/2409.09430/) (2024)
- [Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learning](https://mlsanity.com/paper/2506.05826/) (2025)

## Abstract (arXiv)

In recent years a vast amount of visual content has been generated and shared from many fields, such as social media platforms, medical imaging, and robotics. This abundance of content creation and sharing has introduced new challenges, particularly that of searching databases for similar content-Content Based Image Retrieval (CBIR)-a long-established research area in which improved efficiency and accuracy are needed for real-time retrieval. Artificial intelligence has made progress in CBIR and has significantly facilitated the process of instance search. In this survey we review recent instance retrieval works that are developed based on deep learning algorithms and techniques, with the survey organized by deep network architecture types, deep features, feature embedding and aggregation methods, and network fine-tuning strategies. Our survey considers a wide variety of recent methods, whereby we identify milestone work, reveal connections among various methods and present the commonly used benchmarks, evaluation results, common challenges, and propose promising future directions.