# Fast and Lightweight Scene Regressor for Camera Relocalization

arXiv: https://arxiv.org/abs/2212.01830 · HTML: https://mlsanity.com/paper/2212.01830/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 4 Dec 2022.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [Image-based localization using LSTMs for structured feature correlation](https://mlsanity.com/paper/1611.07890/) (2016)
- [LENS: Localization enhanced by NeRF synthesis](https://mlsanity.com/paper/2110.06558/) (2021)
- [AtLoc: Attention Guided Camera Localization](https://mlsanity.com/paper/1909.03557/) (2019)
- [DFNet: Enhance Absolute Pose Regression with Direct Feature Matching](https://mlsanity.com/paper/2204.00559/) (2022)
- [Expert Sample Consensus Applied to Camera Re-Localization](https://mlsanity.com/paper/1908.02484/) (2019)
- [Learning to Navigate the Energy Landscape](https://mlsanity.com/paper/1603.05772/) (2016)
- [Deep Auxiliary Learning for Visual Localization and Odometry](https://mlsanity.com/paper/1803.03642/) (2018)
- [VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization](https://mlsanity.com/paper/1702.06521/) (2017)
- [KFNet: Learning Temporal Camera Relocalization using Kalman Filtering](https://mlsanity.com/paper/2003.10629/) (2020)
- [Hybrid Scene Compression for Visual Localization](https://mlsanity.com/paper/1807.07512/) (2018)
- [On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation](https://mlsanity.com/paper/1702.02779/) (2017)
- [Leveraging Equivariant Features for Absolute Pose Regression](https://mlsanity.com/paper/2204.02163/) (2022)
- [Efficient Scene Compression for Visual-based Localization](https://mlsanity.com/paper/2011.13894/) (2020)
- [SPP-Net: Deep Absolute Pose Regression with Synthetic Views](https://mlsanity.com/paper/1712.03452/) (2017)

## Built on by (most cited first, 3 of 3)

- [D2S: Representing sparse descriptors and 3D coordinates for camera relocalization](https://mlsanity.com/paper/2307.15250/) (2023)
- [Representing 3D sparse map points and lines for camera relocalization](https://mlsanity.com/paper/2402.18011/) (2024)
- [Improving Efficiency of DNN-based Relocalization Module for Autonomous Driving with Server-side Computing](https://mlsanity.com/paper/2312.00316/) (2023)

## Abstract (arXiv)

Camera relocalization involving a prior 3D reconstruction plays a crucial role in many mixed reality and robotics applications. Estimating the camera pose directly with respect to pre-built 3D models can be prohibitively expensive for several applications with limited storage and/or communication bandwidth. Although recent scene and absolute pose regression methods have become popular for efficient camera localization, most of them are computation-resource intensive and difficult to obtain a real-time inference with high accuracy constraints. This study proposes a simple scene regression method that requires only a multi-layer perceptron network for mapping scene coordinates to achieve accurate camera pose estimations. The proposed approach uses sparse descriptors to regress the scene coordinates, instead of a dense RGB image. The use of sparse features provides several advantages. First, the proposed regressor network is substantially smaller than those reported in previous studies. This makes our system highly efficient and scalable. Second, the pre-built 3D models provide the most reliable and robust 2D-3D matches. Therefore, learning from them can lead to an awareness of equivalent features and substantially improve the generalization performance. A detailed analysis of our approach and extensive evaluations using existing datasets are provided to support the proposed method. The implementation detail is available at https://github.com/aislab/feat2map