# Detect-to-Retrieve: Efficient Regional Aggregation for Image Search

arXiv: https://arxiv.org/abs/1812.01584 · HTML: https://mlsanity.com/paper/1812.01584/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2018.

## Built on by (most cited first, 30 of 39)

- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [Optimal Transport Aggregation for Visual Place Recognition](https://mlsanity.com/paper/2311.15937/) (2023)
- [TransVPR: Transformer-based place recognition with multi-level attention aggregation](https://mlsanity.com/paper/2201.02001/) (2022)
- [Where is your place, Visual Place Recognition?](https://mlsanity.com/paper/2103.06443/) (2021)
- [Deep Learning for Instance Retrieval: A Survey](https://mlsanity.com/paper/2101.11282/) (2021)
- [Instance-level Image Retrieval using Reranking Transformers](https://mlsanity.com/paper/2103.12236/) (2021)
- [DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features](https://mlsanity.com/paper/2108.02927/) (2021)
- [Correlation Verification for Image Retrieval](https://mlsanity.com/paper/2204.01458/) (2022)
- [Global Features are All You Need for Image Retrieval and Reranking](https://mlsanity.com/paper/2308.06954/) (2023)
- [Learning Super-Features for Image Retrieval](https://mlsanity.com/paper/2201.13182/) (2022)
- [Target before Shooting: Accurate Anomaly Detection and Localization under One Millisecond via Cascade Patch Retrieval](https://mlsanity.com/paper/2308.06748/) (2023)
- [IRGen: Generative Modeling for Image Retrieval](https://mlsanity.com/paper/2303.10126/) (2023)
- [StructVPR: Distill Structural Knowledge with Weighting Samples for Visual Place Recognition](https://mlsanity.com/paper/2212.00937/) (2022)
- [All the attention you need: Global-local, spatial-channel attention for image retrieval](https://mlsanity.com/paper/2107.08000/) (2021)
- [InsCLR: Improving Instance Retrieval with Self-Supervision](https://mlsanity.com/paper/2112.01390/) (2021)
- [Enhancing Deformable Local Features by Jointly Learning to Detect and Describe Keypoints](https://mlsanity.com/paper/2304.00583/) (2023)
- [Diagnosing Human-object Interaction Detectors](https://mlsanity.com/paper/2308.08529/) (2023)
- [Extracting Deformation-Aware Local Features by Learning to Deform](https://mlsanity.com/paper/2111.10617/) (2021)
- [A Critical Analysis of Image-based Camera Pose Estimation Techniques](https://mlsanity.com/paper/2201.05816/) (2022)
- [Let All be Whitened: Multi-teacher Distillation for Efficient Visual Retrieval](https://mlsanity.com/paper/2312.09716/) (2023)
- [DALG: Deep Attentive Local and Global Modeling for Image Retrieval](https://mlsanity.com/paper/2207.00287/) (2022)
- [Anomaly Detection of Defect using Energy of Point Pattern Features within Random Finite Set Framework](https://mlsanity.com/paper/2108.12159/) (2021)
- [Tightly Coupled Learning Strategy for Weakly Supervised Hierarchical Place Recognition](https://mlsanity.com/paper/2202.06470/) (2022)
- [Learning Geodesic-Aware Local Features from RGB-D Images](https://mlsanity.com/paper/2203.12016/) (2022)
- [Towards Content-based Pixel Retrieval in Revisited Oxford and Paris](https://mlsanity.com/paper/2309.05438/) (2023)
- [Yes, we CANN: Constrained Approximate Nearest Neighbors for local feature-based visual localization](https://mlsanity.com/paper/2306.09012/) (2023)
- [Coarse-to-Fine: Learning Compact Discriminative Representation for Single-Stage Image Retrieval](https://mlsanity.com/paper/2308.04008/) (2023)
- [PlaceFormer: Transformer-based Visual Place Recognition using Multi-Scale Patch Selection and Fusion](https://mlsanity.com/paper/2401.13082/) (2024)
- [On Train-Test Class Overlap and Detection for Image Retrieval](https://mlsanity.com/paper/2404.01524/) (2024)
- [StructVPR++: Distill Structural and Semantic Knowledge with Weighting Samples for Visual Place Recognition](https://mlsanity.com/paper/2503.06601/) (2025)

## Abstract (arXiv)

Retrieving object instances among cluttered scenes efficiently requires compact yet comprehensive regional image representations. Intuitively, object semantics can help build the index that focuses on the most relevant regions. However, due to the lack of bounding-box datasets for objects of interest among retrieval benchmarks, most recent work on regional representations has focused on either uniform or class-agnostic region selection. In this paper, we first fill the void by providing a new dataset of landmark bounding boxes, based on the Google Landmarks dataset, that includes 86k images with manually curated boxes from 15k unique landmarks. Then, we demonstrate how a trained landmark detector, using our new dataset, can be leveraged to index image regions and improve retrieval accuracy while being much more efficient than existing regional methods. In addition, we introduce a novel regional aggregated selective match kernel (R-ASMK) to effectively combine information from detected regions into an improved holistic image representation. R-ASMK boosts image retrieval accuracy substantially with no dimensionality increase, while even outperforming systems that index image regions independently. Our complete image retrieval system improves upon the previous state-of-the-art by significant margins on the Revisited Oxford and Paris datasets. Code and data available at the project webpage: https://github.com/tensorflow/models/tree/master/research/delf.