# Deep Keypoint-Based Camera Pose Estimation with Geometric Constraints

arXiv: https://arxiv.org/abs/2007.15122 · HTML: https://mlsanity.com/paper/2007.15122/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2020.

## Built on by (most cited first, 29 of 29)

- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [Unsupervised Keypoints from Pretrained Diffusion Models](https://mlsanity.com/paper/2312.00065/) (2023)
- [PnLCalib: Sports Field Registration via Points and Lines Optimization](https://mlsanity.com/paper/2404.08401/) (2024)
- [Learning to Make Keypoints Sub-Pixel Accurate](https://mlsanity.com/paper/2407.11668/) (2024)
- [Learning Accurate Template Matching with Differentiable Coarse-to-Fine Correspondence Refinement](https://mlsanity.com/paper/2303.08438/) (2023)
- [RUBIK: A Structured Benchmark for Image Matching across Geometric Challenges](https://mlsanity.com/paper/2502.19955/) (2025)
- [C3Fusion: Consistent Contrastive Colon Fusion, Towards Deep SLAM in Colonoscopy](https://mlsanity.com/paper/2206.01961/) (2022)
- [Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network](https://mlsanity.com/paper/2207.10506/) (2022)
- [SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learning](https://mlsanity.com/paper/2407.05283/) (2024)
- [SuperPoint features in endoscopy](https://mlsanity.com/paper/2203.04302/) (2022)
- [Vision-based system identification and 3D keypoint discovery using dynamics constraints](https://mlsanity.com/paper/2109.05928/) (2021)
- [MC-NeRF: Multi-Camera Neural Radiance Fields for Multi-Camera Image Acquisition Systems](https://mlsanity.com/paper/2309.07846/) (2023)
- [RING#: PR-by-PE Global Localization with Roto-translation Equivariant Gram Learning](https://mlsanity.com/paper/2409.00206/) (2024)
- [Neural Textured Deformable Meshes for Robust Analysis-by-Synthesis](https://mlsanity.com/paper/2306.00118/) (2023)
- [A Keypoint Detection and Description Network Based on the Vessel Structure for Multi-Modal Retinal Image Registration](https://mlsanity.com/paper/2201.02242/) (2022)
- [AlignDiff: Learning Physically-Grounded Camera Alignment via Diffusion](https://mlsanity.com/paper/2503.21581/) (2025)
- [Bi-directional Loop Closure for Visual SLAM](https://mlsanity.com/paper/2204.01524/) (2022)
- [Semantic SuperPoint: A Deep Semantic Descriptor](https://mlsanity.com/paper/2211.01098/) (2022)
- [DK-SLAM: Monocular Visual SLAM with Deep Keypoint Learning, Tracking and Loop-Closing](https://mlsanity.com/paper/2401.09160/) (2024)
- [Consistent and Optimal Solution to Camera Motion Estimation](https://mlsanity.com/paper/2403.01174/) (2024)
- [Robust Two-View Geometry Estimation with Implicit Differentiation](https://mlsanity.com/paper/2410.17983/) (2024)
- [Looking Beyond Corners: Contrastive Learning of Visual Representations for Keypoint Detection and Description Extraction](https://mlsanity.com/paper/2112.12002/) (2021)
- [DDM-NET: End-to-end learning of keypoint feature Detection, Description and Matching for 3D localization](https://mlsanity.com/paper/2212.04575/) (2022)
- [YOLOPoint Joint Keypoint and Object Detection](https://mlsanity.com/paper/2402.03989/) (2024)
- [A Birotation Solution for Relative Pose Problems](https://mlsanity.com/paper/2505.02025/) (2025)
- [Exploring Category-level Articulated Object Pose Tracking on SE(3) Manifolds](https://mlsanity.com/paper/2511.05996/) (2025)
- [XRefine: Attention-Guided Keypoint Match Refinement](https://mlsanity.com/paper/2601.12530/) (2026)
- [SuperPoint-E: local features for 3D reconstruction via tracking adaptation in endoscopy](https://mlsanity.com/paper/2602.04108/) (2026)
- [Relational Epipolar Graphs for Robust Relative Camera Pose Estimation](https://mlsanity.com/paper/2604.04554/) (2026)

## Abstract (arXiv)

Estimating relative camera poses from consecutive frames is a fundamental problem in visual odometry (VO) and simultaneous localization and mapping (SLAM), where classic methods consisting of hand-crafted features and sampling-based outlier rejection have been a dominant choice for over a decade. Although multiple works propose to replace these modules with learning-based counterparts, most have not yet been as accurate, robust and generalizable as conventional methods. In this paper, we design an end-to-end trainable framework consisting of learnable modules for detection, feature extraction, matching and outlier rejection, while directly optimizing for the geometric pose objective. We show both quantitatively and qualitatively that pose estimation performance may be achieved on par with the classic pipeline. Moreover, we are able to show by end-to-end training, the key components of the pipeline could be significantly improved, which leads to better generalizability to unseen datasets compared to existing learning-based methods.