# Learning and aggregating deep local descriptors for instance-level recognition

arXiv: https://arxiv.org/abs/2007.13172 · HTML: https://mlsanity.com/paper/2007.13172/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2020.

## Built on by (most cited first, 30 of 55)

- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors](https://mlsanity.com/paper/2412.12392/) (2024)
- [A Self-Supervised Descriptor for Image Copy Detection](https://mlsanity.com/paper/2202.10261/) (2022)
- [MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion](https://mlsanity.com/paper/2409.19152/) (2024)
- [ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity](https://mlsanity.com/paper/2203.08101/) (2022)
- [Deep Learning for Instance Retrieval: A Survey](https://mlsanity.com/paper/2101.11282/) (2021)
- [Instance-level Image Retrieval using Reranking Transformers](https://mlsanity.com/paper/2103.12236/) (2021)
- [DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features](https://mlsanity.com/paper/2108.02927/) (2021)
- [Recall@k Surrogate Loss with Large Batches and Similarity Mixup](https://mlsanity.com/paper/2108.11179/) (2021)
- [Correlation Verification for Image Retrieval](https://mlsanity.com/paper/2204.01458/) (2022)
- [The 2021 Image Similarity Dataset and Challenge](https://mlsanity.com/paper/2106.09672/) (2021)
- [Global Features are All You Need for Image Retrieval and Reranking](https://mlsanity.com/paper/2308.06954/) (2023)
- [The Met Dataset: Instance-level Recognition for Artworks](https://mlsanity.com/paper/2202.01747/) (2022)
- [Learning Super-Features for Image Retrieval](https://mlsanity.com/paper/2201.13182/) (2022)
- [Hyperdimensional computing as a framework for systematic aggregation of image descriptors](https://mlsanity.com/paper/2101.07720/) (2021)
- [NinjaDesc: Content-Concealing Visual Descriptors via Adversarial Learning](https://mlsanity.com/paper/2112.12785/) (2021)
- [Are Local Features All You Need for Cross-Domain Visual Place Recognition?](https://mlsanity.com/paper/2304.05887/) (2023)
- [Boosting vision transformers for image retrieval](https://mlsanity.com/paper/2210.11909/) (2022)
- [ARMBench: An Object-centric Benchmark Dataset for Robotic Manipulation](https://mlsanity.com/paper/2303.16382/) (2023)
- [Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit?](https://mlsanity.com/paper/2309.06891/) (2023)
- [AMES: Asymmetric and Memory-Efficient Similarity Estimation for Instance-level Retrieval](https://mlsanity.com/paper/2408.03282/) (2024)
- [Contextual Similarity Aggregation with Self-attention for Visual Re-ranking](https://mlsanity.com/paper/2110.13430/) (2021)
- [SACReg: Scene-Agnostic Coordinate Regression for Visual Localization](https://mlsanity.com/paper/2307.11702/) (2023)
- [All the attention you need: Global-local, spatial-channel attention for image retrieval](https://mlsanity.com/paper/2107.08000/) (2021)
- [Danish Airs and Grounds: A Dataset for Aerial-to-Street-Level Place Recognition and Localization](https://mlsanity.com/paper/2202.01821/) (2022)
- [Graph Convolution Based Efficient Re-Ranking for Visual Retrieval](https://mlsanity.com/paper/2306.08792/) (2023)
- [TLDR: Twin Learning for Dimensionality Reduction](https://mlsanity.com/paper/2110.09455/) (2021)
- [MTLDesc: Looking Wider to Describe Better](https://mlsanity.com/paper/2203.07003/) (2022)
- [DINO-Mix: Enhancing Visual Place Recognition with Foundational Vision Model and Feature Mixing](https://mlsanity.com/paper/2311.00230/) (2023)
- [Let All be Whitened: Multi-teacher Distillation for Efficient Visual Retrieval](https://mlsanity.com/paper/2312.09716/) (2023)

## Abstract (arXiv)

We propose an efficient method to learn deep local descriptors for instance-level recognition. The training only requires examples of positive and negative image pairs and is performed as metric learning of sum-pooled global image descriptors. At inference, the local descriptors are provided by the activations of internal components of the network. We demonstrate why such an approach learns local descriptors that work well for image similarity estimation with classical efficient match kernel methods. The experimental validation studies the trade-off between performance and memory requirements of the state-of-the-art image search approach based on match kernels. Compared to existing local descriptors, the proposed ones perform better in two instance-level recognition tasks and keep memory requirements lower. We experimentally show that global descriptors are not effective enough at large scale and that local descriptors are essential. We achieve state-of-the-art performance, in some cases even with a backbone network as small as ResNet18.