# Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty

arXiv: https://arxiv.org/abs/2104.08278 · HTML: https://mlsanity.com/paper/2104.08278/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 79 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry](https://mlsanity.com/paper/2003.01060/) (2020)
- [DeMoN: Depth and Motion Network for Learning Monocular Stereo](https://mlsanity.com/paper/1612.02401/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [Learning to Find Good Correspondences](https://mlsanity.com/paper/1711.05971/) (2017)
- [BA-Net: Dense Bundle Adjustment Network](https://mlsanity.com/paper/1806.04807/) (2018)
- [Learning Depth from Monocular Videos using Direct Methods](https://mlsanity.com/paper/1712.00175/) (2017)
- [CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM](https://mlsanity.com/paper/1804.00874/) (2018)
- [Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow](https://mlsanity.com/paper/1802.07095/) (2018)
- [Modelling Uncertainty in Deep Learning for Camera Relocalization](https://mlsanity.com/paper/1509.05909/) (2015)
- [DeepSFM: Structure From Motion Via Deep Bundle Adjustment](https://mlsanity.com/paper/1912.09697/) (2019)
- [Pseudo RGB-D for Self-Improving Monocular SLAM and Depth Prediction](https://mlsanity.com/paper/2004.10681/) (2020)
- [Rolling-Shutter-Aware Differential SfM and Image Rectification](https://mlsanity.com/paper/1903.03943/) (2019)
- [Moving Indoor: Unsupervised Video Depth Learning in Challenging Environments](https://mlsanity.com/paper/1910.08898/) (2019)
- [NM-Net: Mining Reliable Neighbors for Robust Feature Correspondences](https://mlsanity.com/paper/1904.00320/) (2019)
- [LS-Net: Learning to Solve Nonlinear Least Squares for Monocular Stereo](https://mlsanity.com/paper/1809.02966/) (2018)
- [Baseline Desensitizing In Translation Averaging](https://mlsanity.com/paper/1901.00643/) (2019)
- [Self-Supervised Human Depth Estimation from Monocular Videos](https://mlsanity.com/paper/2005.03358/) (2020)
- [Calibration Wizard: A Guidance System for Camera Calibration Based on Modelling Geometric and Corner Uncertainty](https://mlsanity.com/paper/1811.03264/) (2018)
- [Minimal Solutions for Relative Pose with a Single Affine Correspondence](https://mlsanity.com/paper/1912.10776/) (2019)

## Built on by (most cited first, 9 of 9)

- [SPARF: Neural Radiance Fields from Sparse and Noisy Poses](https://mlsanity.com/paper/2211.11738/) (2022)
- [Deep Learning for Visual Localization and Mapping: A Survey](https://mlsanity.com/paper/2308.14039/) (2023)
- [DeepAAT: Deep Automated Aerial Triangulation for Fast UAV-based Mapping](https://mlsanity.com/paper/2402.01134/) (2024)
- [Camera Calibration via Circular Patterns: A Comprehensive Framework with Detection Uncertainty and Unbiased Projection Model](https://mlsanity.com/paper/2506.16842/) (2025)
- [Globally Optimal Solution to the Generalized Relative Pose Estimation Problem using Affine Correspondences](https://mlsanity.com/paper/2512.17188/) (2025)
- [A Minimal Solver for Relative Pose Estimation with Unknown Focal Length from Two Affine Correspondences](https://mlsanity.com/paper/2512.22833/) (2025)
- [CameraEditor: Camera-Controlled Image Editing via Video-Prior Sequential Modeling](https://mlsanity.com/paper/2609.01479/) (2026)
- [Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model](https://mlsanity.com/paper/2111.11055/) (2021)
- [Str-L Pose: Integrating Point and Structured Line for Relative Pose Estimation in Dual-Graph](https://mlsanity.com/paper/2408.15750/) (2024)

## Abstract (arXiv)

Learning methods for relative camera pose estimation have been developed largely in isolation from classical geometric approaches. The question of how to integrate predictions from deep neural networks (DNNs) and solutions from geometric solvers, such as the 5-point algorithm, has as yet remained under-explored. In this paper, we present a novel framework that involves probabilistic fusion between the two families of predictions during network training, with a view to leveraging their complementary benefits in a learnable way. The fusion is achieved by learning the DNN uncertainty under explicit guidance by the geometric uncertainty, thereby learning to take into account the geometric solution in relation to the DNN prediction. Our network features a self-attention graph neural network, which drives the learning by enforcing strong interactions between different correspondences and potentially modeling complex relationships between points. We propose motion parmeterizations suitable for learning and show that our method achieves state-of-the-art performance on the challenging DeMoN and ScanNet datasets. While we focus on relative pose, we envision that our pipeline is broadly applicable for fusing classical geometry and deep learning.