# DynaWeightPnP: Toward global real-time 3D-2D solver in PnP without correspondences

arXiv: https://arxiv.org/abs/2409.18457 · HTML: https://mlsanity.com/paper/2409.18457/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 72 references that are in the map.
- First posted on arXiv 27 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [TEASER: Fast and Certifiable Point Cloud Registration](https://mlsanity.com/paper/2001.07715/) (2020)
- [Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration](https://mlsanity.com/paper/1605.03344/) (2016)
- [Guaranteed Outlier Removal for Point Cloud Registration with Correspondences](https://mlsanity.com/paper/1711.10209/) (2017)
- [An Artificial Agent for Robust Image Registration](https://mlsanity.com/paper/1611.10336/) (2016)
- [Dilated FCN for Multi-Agent 2D/3D Medical Image Registration](https://mlsanity.com/paper/1712.01651/) (2017)
- [Fast and Robust Registration of Partially Overlapping Point Clouds](https://mlsanity.com/paper/2112.09922/) (2021)
- [Autonomous Navigation for Robot-assisted Intraluminal and Endovascular Procedures: A Systematic Review](https://mlsanity.com/paper/2305.04027/) (2023)
- [Large-displacement 3D Object Tracking with Hybrid Non-local Optimization](https://mlsanity.com/paper/2207.12620/) (2022)
- [Nonparametric Continuous Sensor Registration](https://mlsanity.com/paper/2001.04286/) (2020)
- [3D Scene Geometry-Aware Constraint for Camera Localization with Deep Learning](https://mlsanity.com/paper/2005.06147/) (2020)
- [Iterative PnP and its application in 3D-2D vascular image registration for robot navigation](https://mlsanity.com/paper/2310.12551/) (2023)
- [Fusing Convolutional Neural Network and Geometric Constraint for Image-based Indoor Localization](https://mlsanity.com/paper/2201.01408/) (2022)
- [RKHS-BA: A Robust Correspondence-Free Multi-View Registration Framework with Semantic Point Clouds](https://mlsanity.com/paper/2403.01254/) (2024)

## Built on by (most cited first, 1 of 1)

- [Advancing Embodied Intelligence in Robotic-Assisted Endovascular Procedures: A Systematic Review of AI Solutions](https://mlsanity.com/paper/2504.15327/) (2025)

## Abstract (arXiv)

This paper addresses a special Perspective-n-Point (PnP) problem: estimating the optimal pose to align 3D and 2D shapes in real-time without correspondences, termed as correspondence-free PnP. While several studies have focused on 3D and 2D shape registration, achieving both real-time and accurate performance remains challenging. This study specifically targets the 3D-2D geometric shape registration tasks, applying the recently developed Reproducing Kernel Hilbert Space (RKHS) to address the "big-to-small" issue. An iterative reweighted least squares method is employed to solve the RKHS-based formulation efficiently. Moreover, our work identifies a unique and interesting observability issue in correspondence-free PnP: the numerical ambiguity between rotation and translation. To address this, we proposed DynaWeightPnP, introducing a dynamic weighting sub-problem and an alternative searching algorithm designed to enhance pose estimation and alignment accuracy. Experiments were conducted on a typical case, that is, a 3D-2D vascular centerline registration task within Endovascular Image-Guided Interventions (EIGIs). Results demonstrated that the proposed algorithm achieves registration processing rates of 60 Hz (without post-refinement) and 31 Hz (with post-refinement) on modern single-core CPUs, with competitive accuracy comparable to existing methods. These results underscore the suitability of DynaWeightPnP for future robot navigation tasks like EIGIs.