# Let's Take This Online: Adapting Scene Coordinate Regression Network Predictions for Online RGB-D Camera Relocalisation

arXiv: https://arxiv.org/abs/1906.08744 · HTML: https://mlsanity.com/paper/1906.08744/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2019.

## Built on by (most cited first, 28 of 28)

- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Poses](https://mlsanity.com/paper/2305.14059/) (2023)
- [Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizer](https://mlsanity.com/paper/2404.14351/) (2024)
- [On the Limits of Pseudo Ground Truth in Visual Camera Re-localisation](https://mlsanity.com/paper/2109.00524/) (2021)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [Is Geometry Enough for Matching in Visual Localization?](https://mlsanity.com/paper/2203.12979/) (2022)
- [GLACE: Global Local Accelerated Coordinate Encoding](https://mlsanity.com/paper/2406.04340/) (2024)
- [Visual Camera Re-Localization Using Graph Neural Networks and Relative Pose Supervision](https://mlsanity.com/paper/2104.02538/) (2021)
- [SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs](https://mlsanity.com/paper/2404.00469/) (2024)
- [Investigating the Role of Image Retrieval for Visual Localization -- An exhaustive benchmark](https://mlsanity.com/paper/2205.15761/) (2022)
- [HSCNet++: Hierarchical Scene Coordinate Classification and Regression for Visual Localization with Transformer](https://mlsanity.com/paper/2305.03595/) (2023)
- [The Unreasonable Effectiveness of Pre-Trained Features for Camera Pose Refinement](https://mlsanity.com/paper/2404.10438/) (2024)
- [Visual Localization using Imperfect 3D Models from the Internet](https://mlsanity.com/paper/2304.05947/) (2023)
- [How Privacy-Preserving are Line Clouds? Recovering Scene Details from 3D Lines](https://mlsanity.com/paper/2103.05086/) (2021)
- [Satellite Image Based Cross-view Localization for Autonomous Vehicle](https://mlsanity.com/paper/2207.13506/) (2022)
- [Continual Learning for Image-Based Camera Localization](https://mlsanity.com/paper/2108.09112/) (2021)
- [FocusTune: Tuning Visual Localization through Focus-Guided Sampling](https://mlsanity.com/paper/2311.02872/) (2023)
- ["Where am I?" Scene Retrieval with Language](https://mlsanity.com/paper/2404.14565/) (2024)
- [CrowdDriven: A New Challenging Dataset for Outdoor Visual Localization](https://mlsanity.com/paper/2109.04527/) (2021)
- [Learning to Produce Semi-dense Correspondences for Visual Localization](https://mlsanity.com/paper/2402.08359/) (2024)
- [A Guide to Structureless Visual Localization](https://mlsanity.com/paper/2504.17636/) (2025)
- [Lazy Visual Localization via Motion Averaging](https://mlsanity.com/paper/2307.09981/) (2023)
- [Privacy-Preserving Representations are not Enough -- Recovering Scene Content from Camera Poses](https://mlsanity.com/paper/2305.04603/) (2023)
- [A Scene is Worth a Thousand Features: Feed-Forward Camera Localization from a Collection of Image Features](https://mlsanity.com/paper/2510.00978/) (2025)
- [Combining Absolute and Semi-Generalized Relative Poses for Visual Localization](https://mlsanity.com/paper/2409.14269/) (2024)
- [Scene Coordinate Reconstruction Priors](https://mlsanity.com/paper/2510.12387/) (2025)
- [Graph Attention Network for Camera Relocalization on Dynamic Scenes](https://mlsanity.com/paper/2209.15056/) (2022)
- [Robust Visual Localization via Semantic-Guided Multi-Scale Transformer](https://mlsanity.com/paper/2506.08526/) (2025)

## Abstract (arXiv)

Many applications require a camera to be relocalised online, without expensive offline training on the target scene. Whilst both keyframe and sparse keypoint matching methods can be used online, the former often fail away from the training trajectory, and the latter can struggle in textureless regions. By contrast, scene coordinate regression (SCoRe) methods generalise to novel poses and can leverage dense correspondences to improve robustness, and recent work has shown how to adapt SCoRe forests between scenes, allowing their state-of-the-art performance to be leveraged online. However, because they use features hand-crafted for indoor use, they do not generalise well to harder outdoor scenes. Whilst replacing the forest with a neural network and learning suitable features for outdoor use is possible, the techniques used to adapt forests between scenes are unfortunately harder to transfer to a network context. In this paper, we address this by proposing a novel way of leveraging a network trained on one scene to predict points in another scene. Our approach replaces the appearance clustering performed by the branching structure of a regression forest with a two-step process that first uses the network to predict points in the original scene, and then uses these predicted points to look up clusters of points from the new scene. We show experimentally that our online approach achieves state-of-the-art performance on both the 7-Scenes and Cambridge Landmarks datasets, whilst running in under 300ms, making it highly effective in live scenarios.