# Geometric Loss Functions for Camera Pose Regression with Deep Learning

arXiv: https://arxiv.org/abs/1704.00390 · HTML: https://mlsanity.com/paper/1704.00390/

## Key facts

- Cited by 224 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2017.

## Built on by (most cited first, 30 of 224)

- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion](https://mlsanity.com/paper/2210.10716/) (2022)
- [GNeRF: GAN-based Neural Radiance Field without Posed Camera](https://mlsanity.com/paper/2103.15606/) (2021)
- [LENS: Localization enhanced by NeRF synthesis](https://mlsanity.com/paper/2110.06558/) (2021)
- [Map-free Visual Relocalization: Metric Pose Relative to a Single Image](https://mlsanity.com/paper/2210.05494/) (2022)
- [Learning Multi-Scene Absolute Pose Regression with Transformers](https://mlsanity.com/paper/2103.11468/) (2021)
- [Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Poses](https://mlsanity.com/paper/2305.14059/) (2023)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [DFNet: Enhance Absolute Pose Regression with Direct Feature Matching](https://mlsanity.com/paper/2204.00559/) (2022)
- [Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization](https://mlsanity.com/paper/2412.08376/) (2024)
- [ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformers](https://mlsanity.com/paper/2203.10157/) (2022)
- [Multi-Frame Self-Supervised Depth with Transformers](https://mlsanity.com/paper/2204.07616/) (2022)
- [On the Limits of Pseudo Ground Truth in Visual Camera Re-localisation](https://mlsanity.com/paper/2109.00524/) (2021)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [Direct-PoseNet: Absolute Pose Regression with Photometric Consistency](https://mlsanity.com/paper/2104.04073/) (2021)
- [Is Geometry Enough for Matching in Visual Localization?](https://mlsanity.com/paper/2203.12979/) (2022)
- [GLACE: Global Local Accelerated Coordinate Encoding](https://mlsanity.com/paper/2406.04340/) (2024)
- [SOLD2: Self-supervised Occlusion-aware Line Description and Detection](https://mlsanity.com/paper/2104.03362/) (2021)
- [Boosting 3-DoF Ground-to-Satellite Camera Localization Accuracy via Geometry-Guided Cross-View Transformer](https://mlsanity.com/paper/2307.08015/) (2023)
- [Map-Relative Pose Regression for Visual Re-Localization](https://mlsanity.com/paper/2404.09884/) (2024)
- [Full Surround Monodepth from Multiple Cameras](https://mlsanity.com/paper/2104.00152/) (2021)
- [3D Line Mapping Revisited](https://mlsanity.com/paper/2303.17504/) (2023)
- [Visual Camera Re-Localization Using Graph Neural Networks and Relative Pose Supervision](https://mlsanity.com/paper/2104.02538/) (2021)
- [Neural Refinement for Absolute Pose Regression with Feature Synthesis](https://mlsanity.com/paper/2303.10087/) (2023)
- [Visual Localization via Few-Shot Scene Region Classification](https://mlsanity.com/paper/2208.06933/) (2022)
- [CoordiNet: uncertainty-aware pose regressor for reliable vehicle localization](https://mlsanity.com/paper/2103.10796/) (2021)
- [DPODv2: Dense Correspondence-Based 6 DoF Pose Estimation](https://mlsanity.com/paper/2207.02805/) (2022)
- [The NeRFect Match: Exploring NeRF Features for Visual Localization](https://mlsanity.com/paper/2403.09577/) (2024)
- [Projective Manifold Gradient Layer for Deep Rotation Regression](https://mlsanity.com/paper/2110.11657/) (2021)

## Abstract (arXiv)

Deep learning has shown to be effective for robust and real-time monocular image relocalisation. In particular, PoseNet is a deep convolutional neural network which learns to regress the 6-DOF camera pose from a single image. It learns to localize using high level features and is robust to difficult lighting, motion blur and unknown camera intrinsics, where point based SIFT registration fails. However, it was trained using a naive loss function, with hyper-parameters which require expensive tuning. In this paper, we give the problem a more fundamental theoretical treatment. We explore a number of novel loss functions for learning camera pose which are based on geometry and scene reprojection error. Additionally we show how to automatically learn an optimal weighting to simultaneously regress position and orientation. By leveraging geometry, we demonstrate that our technique significantly improves PoseNet's performance across datasets ranging from indoor rooms to a small city.