# Reprojection Errors as Prompts for Efficient Scene Coordinate Regression

arXiv: https://arxiv.org/abs/2409.04178 · HTML: https://mlsanity.com/paper/2409.04178/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 6 Sep 2024.

## Builds on (most cited first, 27 of 27)

- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [Vision Transformer Adapter for Dense Predictions](https://mlsanity.com/paper/2205.08534/) (2022)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC](https://mlsanity.com/paper/2002.12324/) (2020)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Poses](https://mlsanity.com/paper/2305.14059/) (2023)
- [Expert Sample Consensus Applied to Camera Re-Localization](https://mlsanity.com/paper/1908.02484/) (2019)
- [Privacy Preserving Image-Based Localization](https://mlsanity.com/paper/1903.05572/) (2019)
- [Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes](https://mlsanity.com/paper/2008.02004/) (2020)
- [Large-scale Localization Datasets in Crowded Indoor Spaces](https://mlsanity.com/paper/2105.08941/) (2021)
- [VS-Net: Voting with Segmentation for Visual Localization](https://mlsanity.com/paper/2105.10886/) (2021)
- [SFD2: Semantic-guided Feature Detection and Description](https://mlsanity.com/paper/2304.14845/) (2023)
- [NeuMap: Neural Coordinate Mapping by Auto-Transdecoder for Camera Localization](https://mlsanity.com/paper/2211.11177/) (2022)
- [Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments](https://mlsanity.com/paper/2012.04746/) (2020)
- [FocusTune: Tuning Visual Localization through Focus-Guided Sampling](https://mlsanity.com/paper/2311.02872/) (2023)
- [Full-Frame Scene Coordinate Regression for Image-Based Localization](https://mlsanity.com/paper/1802.03237/) (2018)
- [On the Instability of Relative Pose Estimation and RANSAC's Role](https://mlsanity.com/paper/2112.14651/) (2021)
- [Scene Coordinate Regression with Angle-Based Reprojection Loss for Camera Relocalization](https://mlsanity.com/paper/1808.04999/) (2018)
- [PixSelect: Less but Reliable Pixels for Accurate and Efficient Localization](https://mlsanity.com/paper/2206.03775/) (2022)
- [Improved Scene Landmark Detection for Camera Localization](https://mlsanity.com/paper/2401.18083/) (2024)
- [OFVL-MS: Once for Visual Localization across Multiple Indoor Scenes](https://mlsanity.com/paper/2308.11928/) (2023)

## Built on by (most cited first, 3 of 3)

- [ACE-G: Improving Generalization of Scene Coordinate Regression Through Query Pre-Training](https://mlsanity.com/paper/2510.11605/) (2025)
- [Scene Coordinate Reconstruction Priors](https://mlsanity.com/paper/2510.12387/) (2025)
- [Enhancing Scene Coordinate Regression with Efficient Keypoint Detection and Sequential Information](https://mlsanity.com/paper/2412.06488/) (2024)

## Abstract (arXiv)

Scene coordinate regression (SCR) methods have emerged as a promising area of research due to their potential for accurate visual localization. However, many existing SCR approaches train on samples from all image regions, including dynamic objects and texture-less areas. Utilizing these areas for optimization during training can potentially hamper the overall performance and efficiency of the model. In this study, we first perform an in-depth analysis to validate the adverse impacts of these areas. Drawing inspiration from our analysis, we then introduce an error-guided feature selection (EGFS) mechanism, in tandem with the use of the Segment Anything Model (SAM). This mechanism seeds low reprojection areas as prompts and expands them into error-guided masks, and then utilizes these masks to sample points and filter out problematic areas in an iterative manner. The experiments demonstrate that our method outperforms existing SCR approaches that do not rely on 3D information on the Cambridge Landmarks and Indoor6 datasets.