# Camera Pose Auto-Encoders for Improving Pose Regression

arXiv: https://arxiv.org/abs/2207.05530 · HTML: https://mlsanity.com/paper/2207.05530/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 48 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC](https://mlsanity.com/paper/2002.12324/) (2020)
- [Geometry-Aware Learning of Maps for Camera Localization](https://mlsanity.com/paper/1712.03342/) (2017)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [Image-based localization using LSTMs for structured feature correlation](https://mlsanity.com/paper/1611.07890/) (2016)
- [Modelling Uncertainty in Deep Learning for Camera Relocalization](https://mlsanity.com/paper/1509.05909/) (2015)
- [Learning Multi-Scene Absolute Pose Regression with Transformers](https://mlsanity.com/paper/2103.11468/) (2021)
- [AtLoc: Attention Guided Camera Localization](https://mlsanity.com/paper/1909.03557/) (2019)
- [Image-based Localization using Hourglass Networks](https://mlsanity.com/paper/1703.07971/) (2017)
- [VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry](https://mlsanity.com/paper/1804.08366/) (2018)
- [To Learn or Not to Learn: Visual Localization from Essential Matrices](https://mlsanity.com/paper/1908.01293/) (2019)
- [Deep Auxiliary Learning for Visual Localization and Odometry](https://mlsanity.com/paper/1803.03642/) (2018)
- [On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation](https://mlsanity.com/paper/1702.02779/) (2017)
- [Visual Camera Re-Localization Using Graph Neural Networks and Relative Pose Supervision](https://mlsanity.com/paper/2104.02538/) (2021)
- [Introduction to Camera Pose Estimation with Deep Learning](https://mlsanity.com/paper/1907.05272/) (2019)
- [Improved Visual Relocalization by Discovering Anchor Points](https://mlsanity.com/paper/1811.04370/) (2018)
- [Efficient Scene Compression for Visual-based Localization](https://mlsanity.com/paper/2011.13894/) (2020)
- [Do We Really Need Scene-specific Pose Encoders?](https://mlsanity.com/paper/2012.12014/) (2020)

## Built on by (most cited first, 28 of 28)

- [Map-Relative Pose Regression for Visual Re-Localization](https://mlsanity.com/paper/2404.09884/) (2024)
- [Neural Refinement for Absolute Pose Regression with Feature Synthesis](https://mlsanity.com/paper/2303.10087/) (2023)
- [HSCNet++: Hierarchical Scene Coordinate Classification and Regression for Visual Localization with Transformer](https://mlsanity.com/paper/2305.03595/) (2023)
- [From Sparse to Dense: Camera Relocalization with Scene-Specific Detector from Feature Gaussian Splatting](https://mlsanity.com/paper/2503.19358/) (2025)
- [Learning Neural Volumetric Pose Features for Camera Localization](https://mlsanity.com/paper/2403.12800/) (2024)
- [Improving Feature-based Visual Localization by Geometry-Aided Matching](https://mlsanity.com/paper/2211.08712/) (2022)
- [Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environments](https://mlsanity.com/paper/2304.07250/) (2023)
- [Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization using Geometrical Information](https://mlsanity.com/paper/2407.15593/) (2024)
- [Adversarial Exploitation of Data Diversity Improves Visual Localization](https://mlsanity.com/paper/2412.00138/) (2024)
- [Learning to Localize in Unseen Scenes with Relative Pose Regressors](https://mlsanity.com/paper/2303.02717/) (2023)
- [EffLoc: Lightweight Vision Transformer for Efficient 6-DOF Camera Relocalization](https://mlsanity.com/paper/2402.13537/) (2024)
- [NaVIP: An Image-Centric Indoor Navigation Solution for Visually Impaired People](https://mlsanity.com/paper/2410.18109/) (2024)
- [Activating Self-Attention for Multi-Scene Absolute Pose Regression](https://mlsanity.com/paper/2411.01443/) (2024)
- [ActLoc: Learning to Localize on the Move via Active Viewpoint Selection](https://mlsanity.com/paper/2508.20981/) (2025)
- [A Lightweight Domain Adaptive Absolute Pose Regressor Using Barlow Twins Objective](https://mlsanity.com/paper/2211.10963/) (2022)
- [VRS-NeRF: Visual Relocalization with Sparse Neural Radiance Field](https://mlsanity.com/paper/2404.09271/) (2024)
- [FaVoR: Features via Voxel Rendering for Camera Relocalization](https://mlsanity.com/paper/2409.07571/) (2024)
- [GRLoc: Geometric Representation Regression for Visual Localization](https://mlsanity.com/paper/2511.13864/) (2025)
- [Estimating Extreme 3D Image Rotation with Transformer Cross-Attention](https://mlsanity.com/paper/2303.02615/) (2023)
- [PRAM: Place Recognition Anywhere Model for Efficient Visual Localization](https://mlsanity.com/paper/2404.07785/) (2024)
- [PoseINN: Realtime Visual-based Pose Regression and Localization with Invertible Neural Networks](https://mlsanity.com/paper/2404.13288/) (2024)
- [HAL-NeRF: High Accuracy Localization Leveraging Neural Radiance Fields](https://mlsanity.com/paper/2504.08901/) (2025)
- [APR-Transformer: Initial Pose Estimation for Localization in Complex Environments through Absolute Pose Regression](https://mlsanity.com/paper/2505.09356/) (2025)
- [Robust Visual Localization via Semantic-Guided Multi-Scale Transformer](https://mlsanity.com/paper/2506.08526/) (2025)
- [Relative Pose Regression with Pose Auto-Encoders: Enhancing Accuracy and Data Efficiency for Retail Applications](https://mlsanity.com/paper/2508.10933/) (2025)
- [Hierarchical Visual Relocalization with Nearest View Synthesis from Feature Gaussian Splatting](https://mlsanity.com/paper/2603.29185/) (2026)
- [PoseCompass: Intelligent Synthetic Pose Selection for Visual Localization](https://mlsanity.com/paper/2605.12144/) (2026)
- [AutoCompass: Accurate Visual Localization on Public Maps by Learning from Weak Labels](https://mlsanity.com/paper/2609.02798/) (2026)

## Abstract (arXiv)

Absolute pose regressor (APR) networks are trained to estimate the pose of the camera given a captured image. They compute latent image representations from which the camera position and orientation are regressed. APRs provide a different tradeoff between localization accuracy, runtime, and memory, compared to structure-based localization schemes that provide state-of-the-art accuracy. In this work, we introduce Camera Pose Auto-Encoders (PAEs), multilayer perceptrons that are trained via a Teacher-Student approach to encode camera poses using APRs as their teachers. We show that the resulting latent pose representations can closely reproduce APR performance and demonstrate their effectiveness for related tasks. Specifically, we propose a light-weight test-time optimization in which the closest train poses are encoded and used to refine camera position estimation. This procedure achieves a new state-of-the-art position accuracy for APRs, on both the CambridgeLandmarks and 7Scenes benchmarks. We also show that train images can be reconstructed from the learned pose encoding, paving the way for integrating visual information from the train set at a low memory cost. Our code and pre-trained models are available at https://github.com/yolish/camera-pose-auto-encoders.