# Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Poses

arXiv: https://arxiv.org/abs/2305.14059 · HTML: https://mlsanity.com/paper/2305.14059/

## Key facts

- Cited by 107 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 73 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 30 of 39)

- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC](https://mlsanity.com/paper/2002.12324/) (2020)
- [Geometry-Aware Learning of Maps for Camera Localization](https://mlsanity.com/paper/1712.03342/) (2017)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses](https://mlsanity.com/paper/1905.04132/) (2019)
- [Map-free Visual Relocalization: Metric Pose Relative to a Single Image](https://mlsanity.com/paper/2210.05494/) (2022)
- [Learning Multi-Scene Absolute Pose Regression with Transformers](https://mlsanity.com/paper/2103.11468/) (2021)
- [Hierarchical Scene Coordinate Classification and Regression for Visual Localization](https://mlsanity.com/paper/1909.06216/) (2019)
- [Expert Sample Consensus Applied to Camera Re-Localization](https://mlsanity.com/paper/1908.02484/) (2019)
- [Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields](https://mlsanity.com/paper/2209.09050/) (2022)
- [Robust Image Retrieval-based Visual Localization using Kapture](https://mlsanity.com/paper/2007.13867/) (2020)
- [On the Limits of Pseudo Ground Truth in Visual Camera Re-localisation](https://mlsanity.com/paper/2109.00524/) (2021)
- [Learning to Navigate the Energy Landscape](https://mlsanity.com/paper/1603.05772/) (2016)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [To Learn or Not to Learn: Visual Localization from Essential Matrices](https://mlsanity.com/paper/1908.01293/) (2019)
- [Is Geometry Enough for Matching in Visual Localization?](https://mlsanity.com/paper/2203.12979/) (2022)
- [Privacy Preserving Image-Based Localization](https://mlsanity.com/paper/1903.05572/) (2019)

## Built on by (most cited first, 30 of 107)

- [3D Reconstruction with Spatial Memory](https://mlsanity.com/paper/2408.16061/) (2024)
- [VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold](https://mlsanity.com/paper/2505.12549/) (2025)
- [Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos](https://mlsanity.com/paper/2412.09621/) (2024)
- [Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization](https://mlsanity.com/paper/2412.08376/) (2024)
- [Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizer](https://mlsanity.com/paper/2404.14351/) (2024)
- [GLACE: Global Local Accelerated Coordinate Encoding](https://mlsanity.com/paper/2406.04340/) (2024)
- [Light3R-SfM: Towards Feed-forward Structure-from-Motion](https://mlsanity.com/paper/2501.14914/) (2025)
- [Map-Relative Pose Regression for Visual Re-Localization](https://mlsanity.com/paper/2404.09884/) (2024)
- [No Pose at All: Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views](https://mlsanity.com/paper/2508.01171/) (2025)
- [The NeRFect Match: Exploring NeRF Features for Visual Localization](https://mlsanity.com/paper/2403.09577/) (2024)
- [GS-CPR: Efficient Camera Pose Refinement via 3D Gaussian Splatting](https://mlsanity.com/paper/2408.11085/) (2024)
- [UAVScenes: A Multi-Modal Dataset for UAVs](https://mlsanity.com/paper/2507.22412/) (2025)
- [General Place Recognition Survey: Towards Real-World Autonomy](https://mlsanity.com/paper/2405.04812/) (2024)
- [SAIL-Recon: Large SfM by Augmenting Scene Regression with Localization](https://mlsanity.com/paper/2508.17972/) (2025)
- [AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend](https://mlsanity.com/paper/2511.20343/) (2025)
- [VGG-T3: Offline Feed-Forward 3D Reconstruction at Scale](https://mlsanity.com/paper/2602.23361/) (2026)
- [R-SCoRe: Revisiting Scene Coordinate Regression for Robust Large-Scale Visual Localization](https://mlsanity.com/paper/2501.01421/) (2025)
- [FocusTune: Tuning Visual Localization through Focus-Guided Sampling](https://mlsanity.com/paper/2311.02872/) (2023)
- [GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splatting for Improved Visual Localization](https://mlsanity.com/paper/2409.16502/) (2024)
- [From Sparse to Dense: Camera Relocalization with Scene-Specific Detector from Feature Gaussian Splatting](https://mlsanity.com/paper/2503.19358/) (2025)
- [Learning to Produce Semi-dense Correspondences for Visual Localization](https://mlsanity.com/paper/2402.08359/) (2024)
- [ZeroGS: Training 3D Gaussian Splatting from Unposed Images](https://mlsanity.com/paper/2411.15779/) (2024)
- [LightLoc: Learning Outdoor LiDAR Localization at Light Speed](https://mlsanity.com/paper/2503.17814/) (2025)
- [Gaussian Splatting Feature Fields for Privacy-Preserving Visual Localization](https://mlsanity.com/paper/2507.23569/) (2025)
- [Differentiable Product Quantization for Memory Efficient Camera Relocalization](https://mlsanity.com/paper/2407.15540/) (2024)
- [A Guide to Structureless Visual Localization](https://mlsanity.com/paper/2504.17636/) (2025)
- [ACE-G: Improving Generalization of Scene Coordinate Regression Through Query Pre-Training](https://mlsanity.com/paper/2510.11605/) (2025)
- [HGSLoc: 3DGS-based Heuristic Camera Pose Refinement](https://mlsanity.com/paper/2409.10925/) (2024)
- [Seeing in the Dark: Benchmarking Egocentric 3D Vision with the Oxford Day-and-Night Dataset](https://mlsanity.com/paper/2506.04224/) (2025)
- [SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views](https://mlsanity.com/paper/2509.17246/) (2025)

## Abstract (arXiv)

Learning-based visual relocalizers exhibit leading pose accuracy, but require hours or days of training. Since training needs to happen on each new scene again, long training times make learning-based relocalization impractical for most applications, despite its promise of high accuracy. In this paper we show how such a system can actually achieve the same accuracy in less than 5 minutes. We start from the obvious: a relocalization network can be split in a scene-agnostic feature backbone, and a scene-specific prediction head. Less obvious: using an MLP prediction head allows us to optimize across thousands of view points simultaneously in each single training iteration. This leads to stable and extremely fast convergence. Furthermore, we substitute effective but slow end-to-end training using a robust pose solver with a curriculum over a reprojection loss. Our approach does not require privileged knowledge, such a depth maps or a 3D model, for speedy training. Overall, our approach is up to 300x faster in mapping than state-of-the-art scene coordinate regression, while keeping accuracy on par.