# SIFT Meets CNN: A Decade Survey of Instance Retrieval

arXiv: https://arxiv.org/abs/1608.01807 · HTML: https://mlsanity.com/paper/1608.01807/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Aug 2016.

## Built on by (most cited first, 30 of 67)

- [T2VLAD: Global-Local Sequence Alignment for Text-Video Retrieval](https://mlsanity.com/paper/2104.10054/) (2021)
- [Deep Learning for Instance Retrieval: A Survey](https://mlsanity.com/paper/2101.11282/) (2021)
- [Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features](https://mlsanity.com/paper/2308.11485/) (2023)
- [Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization](https://mlsanity.com/paper/2211.07394/) (2022)
- [ILIAS: Instance-Level Image retrieval At Scale](https://mlsanity.com/paper/2502.11748/) (2025)
- [Dense Interaction Learning for Video-based Person Re-identification](https://mlsanity.com/paper/2103.09013/) (2021)
- [Semantic-guided Pixel Sampling for Cloth-Changing Person Re-identification](https://mlsanity.com/paper/2107.11522/) (2021)
- [MSINet: Twins Contrastive Search of Multi-Scale Interaction for Object ReID](https://mlsanity.com/paper/2303.07065/) (2023)
- [Contextual Similarity Aggregation with Self-attention for Visual Re-ranking](https://mlsanity.com/paper/2110.13430/) (2021)
- [Visual and Object Geo-localization: A Comprehensive Survey](https://mlsanity.com/paper/2112.15202/) (2021)
- [Recurrence-Enhanced Vision-and-Language Transformers for Robust Multimodal Document Retrieval](https://mlsanity.com/paper/2503.01980/) (2025)
- [Telling the What while Pointing to the Where: Multimodal Queries for Image Retrieval](https://mlsanity.com/paper/2102.04980/) (2021)
- [Investigating the Vision Transformer Model for Image Retrieval Tasks](https://mlsanity.com/paper/2101.03771/) (2021)
- [A Bag-of-Prototypes Representation for Dataset-Level Applications](https://mlsanity.com/paper/2303.13251/) (2023)
- [Contextually Affinitive Neighborhood Refinery for Deep Clustering](https://mlsanity.com/paper/2312.07806/) (2023)
- [Deep Learning for Scene Classification: A Survey](https://mlsanity.com/paper/2101.10531/) (2021)
- [SIFT Matching by Context Exposed](https://mlsanity.com/paper/2106.09584/) (2021)
- [Deep Triplet Hashing Network for Case-based Medical Image Retrieval](https://mlsanity.com/paper/2101.12346/) (2021)
- [A Survey of Orthogonal Moments for Image Representation: Theory, Implementation, and Evaluation](https://mlsanity.com/paper/2103.14799/) (2021)
- [Which Parts Determine the Impression of the Font?](https://mlsanity.com/paper/2103.14216/) (2021)
- [On Efficient Approximate Queries over Machine Learning Models](https://mlsanity.com/paper/2206.02845/) (2022)
- [Content-Based Search for Deep Generative Models](https://mlsanity.com/paper/2210.03116/) (2022)
- [3D reconstruction from spherical images: A review of techniques, applications, and prospects](https://mlsanity.com/paper/2302.04495/) (2023)
- [Sketch-based Medical Image Retrieval](https://mlsanity.com/paper/2303.03633/) (2023)
- [Pose-dIVE: Pose-Diversified Augmentation with Diffusion Model for Person Re-Identification](https://mlsanity.com/paper/2406.16042/) (2024)
- [Shrinking the Semantic Gap: Spatial Pooling of Local Moment Invariants for Copy-Move Forgery Detection](https://mlsanity.com/paper/2207.09135/) (2022)
- [Steerable Equivariant Representation Learning](https://mlsanity.com/paper/2302.11349/) (2023)
- [Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environments](https://mlsanity.com/paper/2304.07250/) (2023)
- [Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval](https://mlsanity.com/paper/2103.12328/) (2021)
- [Coarse-to-Fine: Learning Compact Discriminative Representation for Single-Stage Image Retrieval](https://mlsanity.com/paper/2308.04008/) (2023)

## Abstract (arXiv)

In the early days, content-based image retrieval (CBIR) was studied with global features. Since 2003, image retrieval based on local descriptors (de facto SIFT) has been extensively studied for over a decade due to the advantage of SIFT in dealing with image transformations. Recently, image representations based on the convolutional neural network (CNN) have attracted increasing interest in the community and demonstrated impressive performance. Given this time of rapid evolution, this article provides a comprehensive survey of instance retrieval over the last decade. Two broad categories, SIFT-based and CNN-based methods, are presented. For the former, according to the codebook size, we organize the literature into using large/medium-sized/small codebooks. For the latter, we discuss three lines of methods, i.e., using pre-trained or fine-tuned CNN models, and hybrid methods. The first two perform a single-pass of an image to the network, while the last category employs a patch-based feature extraction scheme. This survey presents milestones in modern instance retrieval, reviews a broad selection of previous works in different categories, and provides insights on the connection between SIFT and CNN-based methods. After analyzing and comparing retrieval performance of different categories on several datasets, we discuss promising directions towards generic and specialized instance retrieval.