# On the Limits of Pseudo Ground Truth in Visual Camera Re-localisation

arXiv: https://arxiv.org/abs/2109.00524 · HTML: https://mlsanity.com/paper/2109.00524/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 92 references that are in the map.
- First posted on arXiv 1 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC](https://mlsanity.com/paper/2002.12324/) (2020)
- [Geometry-Aware Learning of Maps for Camera Localization](https://mlsanity.com/paper/1712.03342/) (2017)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [Image-based localization using LSTMs for structured feature correlation](https://mlsanity.com/paper/1611.07890/) (2016)
- [Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses](https://mlsanity.com/paper/1905.04132/) (2019)
- [Hierarchical Scene Coordinate Classification and Regression for Visual Localization](https://mlsanity.com/paper/1909.06216/) (2019)
- [Expert Sample Consensus Applied to Camera Re-Localization](https://mlsanity.com/paper/1908.02484/) (2019)
- [Robust Image Retrieval-based Visual Localization using Kapture](https://mlsanity.com/paper/2007.13867/) (2020)
- [Learning to Navigate the Energy Landscape](https://mlsanity.com/paper/1603.05772/) (2016)
- [On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation](https://mlsanity.com/paper/1702.02779/) (2017)
- [Real-Time RGB-D Camera Pose Estimation in Novel Scenes using a Relocalisation Cascade](https://mlsanity.com/paper/1810.12163/) (2018)
- [Project AutoVision: Localization and 3D Scene Perception for an Autonomous Vehicle with a Multi-Camera System](https://mlsanity.com/paper/1809.05477/) (2018)
- [Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes](https://mlsanity.com/paper/2008.02004/) (2020)
- [Let's Take This Online: Adapting Scene Coordinate Regression Network Predictions for Online RGB-D Camera Relocalisation](https://mlsanity.com/paper/1906.08744/) (2019)
- [Backtracking Regression Forests for Accurate Camera Relocalization](https://mlsanity.com/paper/1710.07965/) (2017)
- [Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions](https://mlsanity.com/paper/1707.09092/) (2017)

## Built on by (most cited first, 30 of 63)

- [Map-free Visual Relocalization: Metric Pose Relative to a Single Image](https://mlsanity.com/paper/2210.05494/) (2022)
- [Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Poses](https://mlsanity.com/paper/2305.14059/) (2023)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizer](https://mlsanity.com/paper/2404.14351/) (2024)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [GLACE: Global Local Accelerated Coordinate Encoding](https://mlsanity.com/paper/2406.04340/) (2024)
- [Neural Refinement for Absolute Pose Regression with Feature Synthesis](https://mlsanity.com/paper/2303.10087/) (2023)
- [The NeRFect Match: Exploring NeRF Features for Visual Localization](https://mlsanity.com/paper/2403.09577/) (2024)
- [GS-CPR: Efficient Camera Pose Refinement via 3D Gaussian Splatting](https://mlsanity.com/paper/2408.11085/) (2024)
- [Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motion](https://mlsanity.com/paper/2205.02830/) (2022)
- [Investigating the Role of Image Retrieval for Visual Localization -- An exhaustive benchmark](https://mlsanity.com/paper/2205.15761/) (2022)
- [HSCNet++: Hierarchical Scene Coordinate Classification and Regression for Visual Localization with Transformer](https://mlsanity.com/paper/2305.03595/) (2023)
- [The Unreasonable Effectiveness of Pre-Trained Features for Camera Pose Refinement](https://mlsanity.com/paper/2404.10438/) (2024)
- [Visual Localization using Imperfect 3D Models from the Internet](https://mlsanity.com/paper/2304.05947/) (2023)
- [AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend](https://mlsanity.com/paper/2511.20343/) (2025)
- [FocusTune: Tuning Visual Localization through Focus-Guided Sampling](https://mlsanity.com/paper/2311.02872/) (2023)
- [HR-APR: APR-agnostic Framework with Uncertainty Estimation and Hierarchical Refinement for Camera Relocalisation](https://mlsanity.com/paper/2402.14371/) (2024)
- [A Large Scale Homography Benchmark](https://mlsanity.com/paper/2302.09997/) (2023)
- [SC-wLS: Towards Interpretable Feed-forward Camera Re-localization](https://mlsanity.com/paper/2210.12748/) (2022)
- [From Sparse to Dense: Camera Relocalization with Scene-Specific Detector from Feature Gaussian Splatting](https://mlsanity.com/paper/2503.19358/) (2025)
- [CrowdDriven: A New Challenging Dataset for Outdoor Visual Localization](https://mlsanity.com/paper/2109.04527/) (2021)
- [Two-View Geometry Scoring Without Correspondences](https://mlsanity.com/paper/2306.01596/) (2023)
- [Gaussian Splatting Feature Fields for Privacy-Preserving Visual Localization](https://mlsanity.com/paper/2507.23569/) (2025)
- [A Guide to Structureless Visual Localization](https://mlsanity.com/paper/2504.17636/) (2025)
- [ACE-G: Improving Generalization of Scene Coordinate Regression Through Query Pre-Training](https://mlsanity.com/paper/2510.11605/) (2025)
- [GeoAdapt: Self-Supervised Test-Time Adaptation in LiDAR Place Recognition Using Geometric Priors](https://mlsanity.com/paper/2308.04638/) (2023)
- [HGSLoc: 3DGS-based Heuristic Camera Pose Refinement](https://mlsanity.com/paper/2409.10925/) (2024)
- [Eiffel Tower: A Deep-Sea Underwater Dataset for Long-Term Visual Localization](https://mlsanity.com/paper/2305.05301/) (2023)
- [AdaSfM: From Coarse Global to Fine Incremental Adaptive Structure from Motion](https://mlsanity.com/paper/2301.12135/) (2023)
- [Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement](https://mlsanity.com/paper/2406.08463/) (2024)

## Abstract (arXiv)

Benchmark datasets that measure camera pose accuracy have driven progress in visual re-localisation research. To obtain poses for thousands of images, it is common to use a reference algorithm to generate pseudo ground truth. Popular choices include Structure-from-Motion (SfM) and Simultaneous-Localisation-and-Mapping (SLAM) using additional sensors like depth cameras if available. Re-localisation benchmarks thus measure how well each method replicates the results of the reference algorithm. This begs the question whether the choice of the reference algorithm favours a certain family of re-localisation methods. This paper analyzes two widely used re-localisation datasets and shows that evaluation outcomes indeed vary with the choice of the reference algorithm. We thus question common beliefs in the re-localisation literature, namely that learning-based scene coordinate regression outperforms classical feature-based methods, and that RGB-D-based methods outperform RGB-based methods. We argue that any claims on ranking re-localisation methods should take the type of the reference algorithm, and the similarity of the methods to the reference algorithm, into account.