# Fine-tuning CNN Image Retrieval with No Human Annotation

arXiv: https://arxiv.org/abs/1711.02512 · HTML: https://mlsanity.com/paper/1711.02512/

## Key facts

- Cited by 499 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2017.

## Built on by (most cited first, 30 of 499)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Rethinking Spatial Dimensions of Vision Transformers](https://mlsanity.com/paper/2103.16302/) (2021)
- [ResMLP: Feedforward networks for image classification with data-efficient training](https://mlsanity.com/paper/2105.03404/) (2021)
- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [A Self-Supervised Descriptor for Image Copy Detection](https://mlsanity.com/paper/2202.10261/) (2022)
- [Rethinking Visual Geo-localization for Large-Scale Applications](https://mlsanity.com/paper/2204.02287/) (2022)
- [AnyLoc: Towards Universal Visual Place Recognition](https://mlsanity.com/paper/2308.00688/) (2023)
- [MixVPR: Feature Mixing for Visual Place Recognition](https://mlsanity.com/paper/2303.02190/) (2023)
- [CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion](https://mlsanity.com/paper/2210.10716/) (2022)
- [Optimal Transport Aggregation for Visual Place Recognition](https://mlsanity.com/paper/2311.15937/) (2023)
- [MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection](https://mlsanity.com/paper/2112.08935/) (2021)
- [EigenPlaces: Training Viewpoint Robust Models for Visual Place Recognition](https://mlsanity.com/paper/2308.10832/) (2023)
- [TransVPR: Transformer-based place recognition with multi-level attention aggregation](https://mlsanity.com/paper/2201.02001/) (2022)
- [R2Former: Unified Retrieval and Reranking Transformer for Place Recognition](https://mlsanity.com/paper/2304.03410/) (2023)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [GSV-Cities: Toward Appropriate Supervised Visual Place Recognition](https://mlsanity.com/paper/2210.10239/) (2022)
- [Training Vision Transformers for Image Retrieval](https://mlsanity.com/paper/2102.05644/) (2021)
- [Cluster Contrast for Unsupervised Person Re-Identification](https://mlsanity.com/paper/2103.11568/) (2021)
- [Deep Visual Geo-localization Benchmark](https://mlsanity.com/paper/2204.03444/) (2022)
- [ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity](https://mlsanity.com/paper/2203.08101/) (2022)
- [ML-Decoder: Scalable and Versatile Classification Head](https://mlsanity.com/paper/2111.12933/) (2021)
- [Large Scale Visual Food Recognition](https://mlsanity.com/paper/2103.16107/) (2021)
- [Deep Learning for Instance Retrieval: A Survey](https://mlsanity.com/paper/2101.11282/) (2021)
- [Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences](https://mlsanity.com/paper/2108.07422/) (2021)
- [BoQ: A Place is Worth a Bag of Learnable Queries](https://mlsanity.com/paper/2405.07364/) (2024)
- [Instance-level Image Retrieval using Reranking Transformers](https://mlsanity.com/paper/2103.12236/) (2021)

## Abstract (arXiv)

Image descriptors based on activations of Convolutional Neural Networks (CNNs) have become dominant in image retrieval due to their discriminative power, compactness of representation, and search efficiency. Training of CNNs, either from scratch or fine-tuning, requires a large amount of annotated data, where a high quality of annotation is often crucial. In this work, we propose to fine-tune CNNs for image retrieval on a large collection of unordered images in a fully automated manner. Reconstructed 3D models obtained by the state-of-the-art retrieval and structure-from-motion methods guide the selection of the training data. We show that both hard-positive and hard-negative examples, selected by exploiting the geometry and the camera positions available from the 3D models, enhance the performance of particular-object retrieval. CNN descriptor whitening discriminatively learned from the same training data outperforms commonly used PCA whitening. We propose a novel trainable Generalized-Mean (GeM) pooling layer that generalizes max and average pooling and show that it boosts retrieval performance. Applying the proposed method to the VGG network achieves state-of-the-art performance on the standard benchmarks: Oxford Buildings, Paris, and Holidays datasets.