# Evaluating Contrastive Models for Instance-based Image Retrieval

arXiv: https://arxiv.org/abs/2104.14939 · HTML: https://mlsanity.com/paper/2104.14939/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking](https://mlsanity.com/paper/1803.11285/) (2018)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [Particular object retrieval with integral max-pooling of CNN activations](https://mlsanity.com/paper/1511.05879/) (2015)
- [Deep Image Retrieval: Learning global representations for image search](https://mlsanity.com/paper/1604.01325/) (2016)
- [End-to-end Learning of Deep Visual Representations for Image Retrieval](https://mlsanity.com/paper/1610.07940/) (2016)
- [CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples](https://mlsanity.com/paper/1604.02426/) (2016)
- [Efficient Diffusion on Region Manifolds: Recovering Small Objects with Compact CNN Representations](https://mlsanity.com/paper/1611.05113/) (2016)
- [Multi-scale Orderless Pooling of Deep Convolutional Activation Features](https://mlsanity.com/paper/1403.1840/) (2014)
- [Efficient Image Retrieval via Decoupling Diffusion into Online and Offline Processing](https://mlsanity.com/paper/1811.10907/) (2018)
- [Factors of Transferability for a Generic ConvNet Representation](https://mlsanity.com/paper/1406.5774/) (2014)
- [Bags of Local Convolutional Features for Scalable Instance Search](https://mlsanity.com/paper/1604.04653/) (2016)
- [Neural Codes for Image Retrieval](https://mlsanity.com/paper/1404.1777/) (2014)
- [Multiple Measurements and Joint Dimensionality Reduction for Large Scale Image Search with Short Vectors - Extended Version](https://mlsanity.com/paper/1504.03285/) (2015)

## Built on by (most cited first, 4 of 4)

- [ILIAS: Instance-Level Image retrieval At Scale](https://mlsanity.com/paper/2502.11748/) (2025)
- [Motion Aware Self-Supervision for Generic Event Boundary Detection](https://mlsanity.com/paper/2210.05574/) (2022)
- [Dynamic Channel Selection in Self-Supervised Learning](https://mlsanity.com/paper/2207.12065/) (2022)
- [Geometric Analysis of Self-Supervised Vision Representations for Semantic Image Retrieval](https://mlsanity.com/paper/2604.24469/) (2026)

## Abstract (arXiv)

In this work, we evaluate contrastive models for the task of image retrieval. We hypothesise that models that are learned to encode semantic similarity among instances via discriminative learning should perform well on the task of image retrieval, where relevancy is defined in terms of instances of the same object. Through our extensive evaluation, we find that representations from models trained using contrastive methods perform on-par with (and outperforms) a pre-trained supervised baseline trained on the ImageNet labels in retrieval tasks under various configurations. This is remarkable given that the contrastive models require no explicit supervision. Thus, we conclude that these models can be used to bootstrap base models to build more robust image retrieval engines.