# Coarse-to-Fine Multi-Scene Pose Regression with Transformers

arXiv: https://arxiv.org/abs/2308.11783 · HTML: https://mlsanity.com/paper/2308.11783/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC](https://mlsanity.com/paper/2002.12324/) (2020)
- [Geometry-Aware Learning of Maps for Camera Localization](https://mlsanity.com/paper/1712.03342/) (2017)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [Image-based localization using LSTMs for structured feature correlation](https://mlsanity.com/paper/1611.07890/) (2016)
- [Modelling Uncertainty in Deep Learning for Camera Relocalization](https://mlsanity.com/paper/1509.05909/) (2015)
- [Rank consistent ordinal regression for neural networks with application to age estimation](https://mlsanity.com/paper/1901.07884/) (2019)
- [Learning Multi-Scene Absolute Pose Regression with Transformers](https://mlsanity.com/paper/2103.11468/) (2021)
- [AtLoc: Attention Guided Camera Localization](https://mlsanity.com/paper/1909.03557/) (2019)
- [Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network](https://mlsanity.com/paper/1707.09733/) (2017)
- [Expert Sample Consensus Applied to Camera Re-Localization](https://mlsanity.com/paper/1908.02484/) (2019)
- [VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry](https://mlsanity.com/paper/1804.08366/) (2018)
- [To Learn or Not to Learn: Visual Localization from Essential Matrices](https://mlsanity.com/paper/1908.01293/) (2019)
- [Deep Auxiliary Learning for Visual Localization and Odometry](https://mlsanity.com/paper/1803.03642/) (2018)
- [Visual Camera Re-Localization Using Graph Neural Networks and Relative Pose Supervision](https://mlsanity.com/paper/2104.02538/) (2021)
- [Introduction to Camera Pose Estimation with Deep Learning](https://mlsanity.com/paper/1907.05272/) (2019)
- [Local Supports Global: Deep Camera Relocalization with Sequence Enhancement](https://mlsanity.com/paper/1908.04391/) (2019)

## Built on by (most cited first, 11 of 11)

- [Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization](https://mlsanity.com/paper/2412.08376/) (2024)
- [Differentiable Product Quantization for Memory Efficient Camera Relocalization](https://mlsanity.com/paper/2407.15540/) (2024)
- [RDPN6D: Residual-based Dense Point-wise Network for 6Dof Object Pose Estimation Based on RGB-D Images](https://mlsanity.com/paper/2405.08483/) (2024)
- [Reloc-VGGT: Visual Re-localization with Geometry Grounded Transformer](https://mlsanity.com/paper/2512.21883/) (2025)
- [Activating Self-Attention for Multi-Scene Absolute Pose Regression](https://mlsanity.com/paper/2411.01443/) (2024)
- [ConDo: Continual Domain Expansion for Absolute Pose Regression](https://mlsanity.com/paper/2412.13452/) (2024)
- [MVL-Loc: Leveraging Vision-Language Model for Generalizable Multi-Scene Camera Relocalization](https://mlsanity.com/paper/2507.04509/) (2025)
- [MambaLoc: Efficient Camera Localisation via State Space Model](https://mlsanity.com/paper/2408.09680/) (2024)
- [Disambiguating 2D-3D Correspondences in Gaussian Splatting-based Feature Fields for Visual Localization](https://mlsanity.com/paper/2605.07351/) (2026)
- [Learning 1-Bit LiDAR-based Localization with Auxiliary Objective](https://mlsanity.com/paper/2606.27729/) (2026)
- [LightLoc++: Sensor-Robust Representation Learning for Efficient Outdoor LiDAR Localization](https://mlsanity.com/paper/2608.15317/) (2026)

## Abstract (arXiv)

Absolute camera pose regressors estimate the position and orientation of a camera given the captured image alone. Typically, a convolutional backbone with a multi-layer perceptron (MLP) head is trained using images and pose labels to embed a single reference scene at a time. Recently, this scheme was extended to learn multiple scenes by replacing the MLP head with a set of fully connected layers. In this work, we propose to learn multi-scene absolute camera pose regression with Transformers, where encoders are used to aggregate activation maps with self-attention and decoders transform latent features and scenes encoding into pose predictions. This allows our model to focus on general features that are informative for localization, while embedding multiple scenes in parallel. We extend our previous MS-Transformer approach citeshavit2021learning by introducing a mixed classification-regression architecture that improves the localization accuracy. Our method is evaluated on commonly benchmark indoor and outdoor datasets and has been shown to exceed both multi-scene and state-of-the-art single-scene absolute pose regressors.