# PRAGO: Differentiable Multi-View Pose Optimization From Objectness Detections

arXiv: https://arxiv.org/abs/2403.08586 · HTML: https://mlsanity.com/paper/2403.08586/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 27 of 27)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [Learning Open-World Object Proposals without Learning to Classify](https://mlsanity.com/paper/2108.06753/) (2021)
- [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018)
- [Fusion++: Volumetric Object-Level SLAM](https://mlsanity.com/paper/1808.08378/) (2018)
- [QuadricSLAM: Dual Quadrics from Object Detections as Landmarks in Object-oriented SLAM](https://mlsanity.com/paper/1804.04011/) (2018)
- [Extreme Rotation Estimation using Dense Correlation Volumes](https://mlsanity.com/paper/2104.13530/) (2021)
- [Robust Camera Location Estimation by Convex Programming](https://mlsanity.com/paper/1412.0165/) (2014)
- [NeuRoRA: Neural Robust Rotation Averaging](https://mlsanity.com/paper/1912.04485/) (2019)
- [Baseline Desensitizing In Translation Averaging](https://mlsanity.com/paper/1901.00643/) (2019)
- [HARA: A Hierarchical Approach for Robust Rotation Averaging](https://mlsanity.com/paper/2111.08831/) (2021)
- [Hybrid Rotation Averaging: A Fast and Robust Rotation Averaging Approach](https://mlsanity.com/paper/2101.09116/) (2021)
- [Revisiting Rotation Averaging: Uncertainties and Robust Losses](https://mlsanity.com/paper/2303.05195/) (2023)
- [ShapeFit and ShapeKick for Robust, Scalable Structure from Motion](https://mlsanity.com/paper/1608.02165/) (2016)
- [Deep Permutation Equivariant Structure from Motion](https://mlsanity.com/paper/2104.06703/) (2021)
- [Learning Iterative Robust Transformation Synchronization](https://mlsanity.com/paper/2111.00728/) (2021)
- [Rotation-Only Bundle Adjustment](https://mlsanity.com/paper/2011.11724/) (2020)
- [Two-View Geometry Scoring Without Correspondences](https://mlsanity.com/paper/2306.01596/) (2023)
- [RAGO: Recurrent Graph Optimizer For Multiple Rotation Averaging](https://mlsanity.com/paper/2212.07211/) (2022)
- [Real-time Monocular Object SLAM](https://mlsanity.com/paper/1504.02398/) (2015)
- [Rotation Synchronization via Deep Matrix Factorization](https://mlsanity.com/paper/2305.05268/) (2023)
- [PoserNet: Refining Relative Camera Poses Exploiting Object Detections](https://mlsanity.com/paper/2207.09445/) (2022)
- [Relative Geometry-Aware Siamese Neural Network for 6DOF Camera Relocalization](https://mlsanity.com/paper/1901.01049/) (2019)

## Abstract (arXiv)

Robustly estimating camera poses from a set of images is a fundamental task which remains challenging for differentiable methods, especially in the case of small and sparse camera pose graphs. To overcome this challenge, we propose Pose-refined Rotation Averaging Graph Optimization (PRAGO). From a set of objectness detections on unordered images, our method reconstructs the rotational pose, and in turn, the absolute pose, in a differentiable manner benefiting from the optimization of a sequence of geometrical tasks. We show how our objectness pose-refinement module in PRAGO is able to refine the inherent ambiguities in pairwise relative pose estimation without removing edges and avoiding making early decisions on the viability of graph edges. PRAGO then refines the absolute rotations through iterative graph construction, reweighting the graph edges to compute the final rotational pose, which can be converted into absolute poses using translation averaging. We show that PRAGO is able to outperform non-differentiable solvers on small and sparse scenes extracted from 7-Scenes achieving a relative improvement of 21% for rotations while achieving similar translation estimates.