# Building Rome with Convex Optimization

arXiv: https://arxiv.org/abs/2502.04640 · HTML: https://mlsanity.com/paper/2502.04640/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 7 Feb 2025.

## Builds on (most cited first, 30 of 31)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://mlsanity.com/paper/2410.02073/) (2024)
- [UniDepth: Universal Monocular Metric Depth Estimation](https://mlsanity.com/paper/2403.18913/) (2024)
- [TEASER: Fast and Certifiable Point Cloud Registration](https://mlsanity.com/paper/2001.07715/) (2020)
- [Global Structure-from-Motion Revisited](https://mlsanity.com/paper/2407.20219/) (2024)
- [Manopt, a Matlab toolbox for optimization on manifolds](https://mlsanity.com/paper/1308.5200/) (2013)
- [SE-Sync: A Certifiably Correct Algorithm for Synchronization over the Special Euclidean Group](https://mlsanity.com/paper/1612.07386/) (2016)
- [Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registration](https://mlsanity.com/paper/2206.08903/) (2022)
- [Shonan Rotation Averaging: Global Optimality by Surfing SO(p)n](https://mlsanity.com/paper/2008.02737/) (2020)
- [Outlier-Robust Estimation: Hardness, Minimally Tuned Algorithms, and Applications](https://mlsanity.com/paper/2007.15109/) (2020)
- [Certifiably Optimal Outlier-Robust Geometric Perception: Semidefinite Relaxations and Scalable Global Optimization](https://mlsanity.com/paper/2109.03349/) (2021)
- [Lagrangian Duality in 3D SLAM: Verification Techniques and Optimal Solutions](https://mlsanity.com/paper/1506.00746/) (2015)
- [Global registration of multiple point clouds using semidefinite programming](https://mlsanity.com/paper/1306.5226/) (2013)
- [Distributed Certifiably Correct Pose-Graph Optimization](https://mlsanity.com/paper/1911.03721/) (2019)
- [Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos](https://mlsanity.com/paper/2403.17915/) (2024)
- [MegBA: A GPU-Based Distributed Library for Large-Scale Bundle Adjustment](https://mlsanity.com/paper/2112.01349/) (2021)
- [Certifiable Relative Pose Estimation](https://mlsanity.com/paper/2003.13732/) (2020)
- [An Efficient Global Optimality Certificate for Landmark-Based SLAM](https://mlsanity.com/paper/2206.12961/) (2022)
- [Certifiably Correct Range-Aided SLAM](https://mlsanity.com/paper/2302.11614/) (2023)
- [Certifiably Optimal Rotation and Pose Estimation Based on the Cayley Map](https://mlsanity.com/paper/2308.12418/) (2023)
- [SDPRLayers: Certifiable Backpropagation Through Polynomial Optimization Problems in Robotics](https://mlsanity.com/paper/2405.19309/) (2024)
- [SIM-Sync: From Certifiably Optimal Synchronization over the 3D Similarity Group to Scene Reconstruction with Learned Depth](https://mlsanity.com/paper/2309.05184/) (2023)
- [Fast and Certifiable Trajectory Optimization](https://mlsanity.com/paper/2406.05846/) (2024)

## Built on by (most cited first, 8 of 8)

- [Advances in Global Solvers for 3D Vision](https://mlsanity.com/paper/2602.14662/) (2026)
- [Regist3R: Incremental Registration with Stereo Foundation Model](https://mlsanity.com/paper/2504.12356/) (2025)
- [InstantSfM: Towards GPU-Native SfM for the Deep Learning Era](https://mlsanity.com/paper/2510.13310/) (2025)
- [Sparse Variable Projection in Robotic Perception: Exploiting Separable Structure for Efficient Nonlinear Optimization](https://mlsanity.com/paper/2512.07969/) (2025)
- [ProBA: Probabilistic Bundle Adjustment with the Bhattacharyya Coefficient](https://mlsanity.com/paper/2505.20858/) (2025)
- [Certifiable Factor Graph Optimization](https://mlsanity.com/paper/2603.01267/) (2026)
- [Tempered Sequential Monte Carlo for Trajectory and Policy Optimization with Differentiable Dynamics](https://mlsanity.com/paper/2604.21456/) (2026)
- [Time-to-Collision Based Dynamic Obstacle Avoidance Using Pretrained Vision Models for Robots in Unstructured Environments](https://mlsanity.com/paper/2607.07885/) (2026)

## Abstract (arXiv)

Global bundle adjustment is made easy by depth prediction and convex optimization. We (i) propose a scaled bundle adjustment (SBA) formulation that lifts 2D keypoint measurements to 3D with learned depth, (ii) design an empirically tight convex semidfinite program (SDP) relaxation that solves SBA to certfiable global optimality, (iii) solve the SDP relaxations at extreme scale with Burer-Monteiro factorization and a CUDA-based trust-region Riemannian optimizer (dubbed XM), (iv) build a structure from motion (SfM) pipeline with XM as the optimization engine and show that XM-SfM compares favorably with existing pipelines in terms of reconstruction quality while being significantly faster, more scalable, and initialization-free.