# Benchmarking Image Retrieval for Visual Localization

arXiv: https://arxiv.org/abs/2011.11946 · HTML: https://mlsanity.com/paper/2011.11946/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2020.

## Built on by (most cited first, 30 of 42)

- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [AnyLoc: Towards Universal Visual Place Recognition](https://mlsanity.com/paper/2308.00688/) (2023)
- [Optimal Transport Aggregation for Visual Place Recognition](https://mlsanity.com/paper/2311.15937/) (2023)
- [Map-free Visual Relocalization: Metric Pose Relative to a Single Image](https://mlsanity.com/paper/2210.05494/) (2022)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [Deep Visual Geo-localization Benchmark](https://mlsanity.com/paper/2204.03444/) (2022)
- [Where is your place, Visual Place Recognition?](https://mlsanity.com/paper/2103.06443/) (2021)
- [Data-efficient Large Scale Place Recognition with Graded Similarity Supervision](https://mlsanity.com/paper/2303.11739/) (2023)
- [OpenStreetView-5M: The Many Roads to Global Visual Geolocation](https://mlsanity.com/paper/2404.18873/) (2024)
- [Large-scale Localization Datasets in Crowded Indoor Spaces](https://mlsanity.com/paper/2105.08941/) (2021)
- [SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs](https://mlsanity.com/paper/2404.00469/) (2024)
- [Generalized Contrastive Optimization of Siamese Networks for Place Recognition](https://mlsanity.com/paper/2103.06638/) (2021)
- [Investigating the Role of Image Retrieval for Visual Localization -- An exhaustive benchmark](https://mlsanity.com/paper/2205.15761/) (2022)
- [Are Local Features All You Need for Cross-Domain Visual Place Recognition?](https://mlsanity.com/paper/2304.05887/) (2023)
- [The Unreasonable Effectiveness of Pre-Trained Features for Camera Pose Refinement](https://mlsanity.com/paper/2404.10438/) (2024)
- [Visual Localization using Imperfect 3D Models from the Internet](https://mlsanity.com/paper/2304.05947/) (2023)
- [Reassessing the Limitations of CNN Methods for Camera Pose Regression](https://mlsanity.com/paper/2108.07260/) (2021)
- [Revisit Anything: Visual Place Recognition via Image Segment Retrieval](https://mlsanity.com/paper/2409.18049/) (2024)
- [On the Estimation of Image-matching Uncertainty in Visual Place Recognition](https://mlsanity.com/paper/2404.00546/) (2024)
- ["Where am I?" Scene Retrieval with Language](https://mlsanity.com/paper/2404.14565/) (2024)
- [PICCOLO: Point Cloud-Centric Omnidirectional Localization](https://mlsanity.com/paper/2108.06545/) (2021)
- [Long-term Visual Localization with Mobile Sensors](https://mlsanity.com/paper/2304.07691/) (2023)
- [Learning Semantics for Visual Place Recognition through Multi-Scale Attention](https://mlsanity.com/paper/2201.09701/) (2022)
- [A Guide to Structureless Visual Localization](https://mlsanity.com/paper/2504.17636/) (2025)
- [JIST: Joint Image and Sequence Training for Sequential Visual Place Recognition](https://mlsanity.com/paper/2403.19787/) (2024)
- [DnD: Dense Depth Estimation in Crowded Dynamic Indoor Scenes](https://mlsanity.com/paper/2108.05615/) (2021)
- [Improving Worst Case Visual Localization Coverage via Place-specific Sub-selection in Multi-camera Systems](https://mlsanity.com/paper/2206.13883/) (2022)
- [Self-Supervised Learning for Place Representation Generalization across Appearance Changes](https://mlsanity.com/paper/2303.02370/) (2023)
- [Yes, we CANN: Constrained Approximate Nearest Neighbors for local feature-based visual localization](https://mlsanity.com/paper/2306.09012/) (2023)
- [Improved Scene Landmark Detection for Camera Localization](https://mlsanity.com/paper/2401.18083/) (2024)

## Abstract (arXiv)

Visual localization, i.e., camera pose estimation in a known scene, is a core component of technologies such as autonomous driving and augmented reality. State-of-the-art localization approaches often rely on image retrieval techniques for one of two tasks: (1) provide an approximate pose estimate or (2) determine which parts of the scene are potentially visible in a given query image. It is common practice to use state-of-the-art image retrieval algorithms for these tasks. These algorithms are often trained for the goal of retrieving the same landmark under a large range of viewpoint changes. However, robustness to viewpoint changes is not necessarily desirable in the context of visual localization. This paper focuses on understanding the role of image retrieval for multiple visual localization tasks. We introduce a benchmark setup and compare state-of-the-art retrieval representations on multiple datasets. We show that retrieval performance on classical landmark retrieval/recognition tasks correlates only for some but not all tasks to localization performance. This indicates a need for retrieval approaches specifically designed for localization tasks. Our benchmark and evaluation protocols are available at https://github.com/naver/kapture-localization.