Building Rome with Convex Optimization
Not on the map. Why?
Abstract (arXiv)
Global bundle adjustment is made easy by depth prediction and convex optimization. We (i) propose a scaled bundle adjustment (SBA) formulation that lifts 2D keypoint measurements to 3D with learned depth, (ii) design an empirically tight convex semidfinite program (SDP) relaxation that solves SBA to certfiable global optimality, (iii) solve the SDP relaxations at extreme scale with Burer-Monteiro factorization and a CUDA-based trust-region Riemannian optimizer (dubbed XM), (iv) build a structure from motion (SfM) pipeline with XM as the optimization engine and show that XM-SfM compares favorably with existing pipelines in terms of reconstruction quality while being significantly faster, more scalable, and initialization-free.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 7 Feb 2025.
Builds on
31 of 49 references.
References · 31
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
- 2024Depth Anything V2cited by 1,385
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
- 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 667
- 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
- 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
- 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
- 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
- 2020TEASER: Fast and Certifiable Point Cloud Registrationcited by 274
- 2024Global Structure-from-Motion Revisitedcited by 215
19 more
- 2013Manopt, a Matlab toolbox for optimization on manifoldscited by 112
- 2016SE-Sync: A Certifiably Correct Algorithm for Synchronization over the Special Euclidean Groupcited by 80
- 2022Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registrationcited by 78
- 2020Shonan Rotation Averaging: Global Optimality by Surfing SO(p)ncited by 46
- 2020Outlier-Robust Estimation: Hardness, Minimally Tuned Algorithms, and Applicationscited by 26
- 2021Certifiably Optimal Outlier-Robust Geometric Perception: Semidefinite Relaxations and Scalable Global Optimizationcited by 21
- 2015Lagrangian Duality in 3D SLAM: Verification Techniques and Optimal Solutionscited by 18
- 2013Global registration of multiple point clouds using semidefinite programmingcited by 17
- 2019Distributed Certifiably Correct Pose-Graph Optimizationcited by 17
- 2024Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videoscited by 14
- 2021MegBA: A GPU-Based Distributed Library for Large-Scale Bundle Adjustmentcited by 13
- 2020Certifiable Relative Pose Estimationcited by 11
- 2022An Efficient Global Optimality Certificate for Landmark-Based SLAMcited by 9
- 2023Certifiably Correct Range-Aided SLAMcited by 7
- 2023Certifiably Optimal Rotation and Pose Estimation Based on the Cayley Mapcited by 6
- 2024SDPRLayers: Certifiable Backpropagation Through Polynomial Optimization Problems in Roboticscited by 6
- 2023SIM-Sync: From Certifiably Optimal Synchronization over the 3D Similarity Group to Scene Reconstruction with Learned Depthcited by 5
- 2024Fast and Certifiable Trajectory Optimizationcited by 5
- 2023Decentralization and Acceleration Enables Large-Scale Bundle Adjustmentcited by 3
Built on by
Citing papers per year: 2025: 4; 2026: 4.
8 papers.
- 2026Advances in Global Solvers for 3D Visioncited by 12
- 2025Regist3R: Incremental Registration with Stereo Foundation Modelcited by 9
- 2025InstantSfM: Towards GPU-Native SfM for the Deep Learning Eracited by 4
- 2025Sparse Variable Projection in Robotic Perception: Exploiting Separable Structure for Efficient Nonlinear Optimizationcited by 1
- 2025ProBA: Probabilistic Bundle Adjustment with the Bhattacharyya Coefficientcited by 0
- 2026Certifiable Factor Graph Optimizationcited by 0
- 2026Tempered Sequential Monte Carlo for Trajectory and Policy Optimization with Differentiable Dynamicscited by 0
- 2026Time-to-Collision Based Dynamic Obstacle Avoidance Using Pretrained Vision Models for Robots in Unstructured Environmentscited by 0
- 2026Time-to-Collision Based Dynamic Obstacle Avoidance Using Pretrained Vision Models for Robots in Unstructured Environments8 Jul 2026
- 2026Tempered Sequential Monte Carlo for Trajectory and Policy Optimization with Differentiable Dynamics23 Apr 2026
- 2026Certifiable Factor Graph Optimization1 Mar 2026
- 2026Advances in Global Solvers for 3D Vision16 Feb 2026
- 2025Sparse Variable Projection in Robotic Perception: Exploiting Separable Structure for Efficient Nonlinear Optimization8 Dec 2025
- 2025InstantSfM: Towards GPU-Native SfM for the Deep Learning Era15 Oct 2025
- 2025ProBA: Probabilistic Bundle Adjustment with the Bhattacharyya Coefficient27 May 2025
- 2025Regist3R: Incremental Registration with Stereo Foundation Model16 Apr 2025