# Reassessing the Limitations of CNN Methods for Camera Pose Regression

arXiv: https://arxiv.org/abs/2108.07260 · HTML: https://mlsanity.com/paper/2108.07260/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 71 references that are in the map.
- First posted on arXiv 16 Aug 2021.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Kornia: an Open Source Differentiable Computer Vision Library for PyTorch](https://mlsanity.com/paper/1910.02190/) (2019)
- [Free View Synthesis](https://mlsanity.com/paper/2008.05511/) (2020)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC](https://mlsanity.com/paper/2002.12324/) (2020)
- [Geometry-Aware Learning of Maps for Camera Localization](https://mlsanity.com/paper/1712.03342/) (2017)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [Image-based localization using LSTMs for structured feature correlation](https://mlsanity.com/paper/1611.07890/) (2016)
- [Modelling Uncertainty in Deep Learning for Camera Relocalization](https://mlsanity.com/paper/1509.05909/) (2015)
- [3D Ken Burns Effect from a Single Image](https://mlsanity.com/paper/1909.05483/) (2019)
- [Extreme View Synthesis](https://mlsanity.com/paper/1812.04777/) (2018)
- [Hierarchical Scene Coordinate Classification and Regression for Visual Localization](https://mlsanity.com/paper/1909.06216/) (2019)

## Built on by (most cited first, 19 of 19)

- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [DFNet: Enhance Absolute Pose Regression with Direct Feature Matching](https://mlsanity.com/paper/2204.00559/) (2022)
- [ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformers](https://mlsanity.com/paper/2203.10157/) (2022)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality](https://mlsanity.com/paper/2409.14067/) (2024)
- [360Loc: A Dataset and Benchmark for Omnidirectional Visual Localization with Cross-device Queries](https://mlsanity.com/paper/2311.17389/) (2023)
- [EgoGen: An Egocentric Synthetic Data Generator](https://mlsanity.com/paper/2401.08739/) (2024)
- [Visual Localization using Imperfect 3D Models from the Internet](https://mlsanity.com/paper/2304.05947/) (2023)
- [HR-APR: APR-agnostic Framework with Uncertainty Estimation and Hierarchical Refinement for Camera Relocalisation](https://mlsanity.com/paper/2402.14371/) (2024)
- [A Probabilistic Framework for Visual Localization in Ambiguous Scenes](https://mlsanity.com/paper/2301.02086/) (2023)
- [Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization](https://mlsanity.com/paper/2310.06984/) (2023)
- [NeuraLoc: Visual Localization in Neural Implicit Map with Dual Complementary Features](https://mlsanity.com/paper/2503.06117/) (2025)
- [Leveraging Neural Radiance Field in Descriptor Synthesis for Keypoints Scene Coordinate Regression](https://mlsanity.com/paper/2403.10297/) (2024)
- [Conditional Variational Autoencoders for Probabilistic Pose Regression](https://mlsanity.com/paper/2410.04989/) (2024)
- [MobileARLoc: On-device Robust Absolute Localisation for Pervasive Markerless Mobile AR](https://mlsanity.com/paper/2401.11511/) (2024)
- [KS-APR: Keyframe Selection for Robust Absolute Pose Regression](https://mlsanity.com/paper/2308.05459/) (2023)
- [A-SCoRe: Attention-based Scene Coordinate Regression for wide-ranging scenarios](https://mlsanity.com/paper/2503.13982/) (2025)
- [Quantifying Epistemic Uncertainty in Absolute Pose Regression](https://mlsanity.com/paper/2504.07260/) (2025)
- [SGLoc: Semantic Localization System for Camera Pose Estimation from 3D Gaussian Splatting Representation](https://mlsanity.com/paper/2507.12027/) (2025)

## Abstract (arXiv)

In this paper, we address the problem of camera pose estimation in outdoor and indoor scenarios. In comparison to the currently top-performing methods that rely on 2D to 3D matching, we propose a model that can directly regress the camera pose from images with significantly higher accuracy than existing methods of the same class. We first analyse why regression methods are still behind the state-of-the-art, and we bridge the performance gap with our new approach. Specifically, we propose a way to overcome the biased training data by a novel training technique, which generates poses guided by a probability distribution from the training set for synthesising new training views. Lastly, we evaluate our approach on two widely used benchmarks and show that it achieves significantly improved performance compared to prior regression-based methods, retrieval techniques as well as 3D pipelines with local feature matching.