# Bundle Adjustment in the Eager Mode

arXiv: https://arxiv.org/abs/2409.12190 · HTML: https://mlsanity.com/paper/2409.12190/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 86 references that are in the map.
- First posted on arXiv 18 Sep 2024.

## Builds on (most cited first, 30 of 32)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Visual Geometry Grounded Deep Structure From Motion](https://mlsanity.com/paper/2312.04563/) (2023)
- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality](https://mlsanity.com/paper/2401.16663/) (2024)
- [DKM: Dense Kernelized Feature Matching for Geometry Estimation](https://mlsanity.com/paper/2202.00667/) (2022)
- [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer](https://mlsanity.com/paper/2208.14201/) (2022)
- [PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment](https://mlsanity.com/paper/2306.15667/) (2023)
- [A micro Lie theory for state estimation in robotics](https://mlsanity.com/paper/1812.01537/) (2018)
- [Learning Feature Descriptors using Camera Pose Supervision](https://mlsanity.com/paper/2004.13324/) (2020)
- [Detector-Free Structure from Motion](https://mlsanity.com/paper/2306.15669/) (2023)
- [Theseus: A Library for Differentiable Nonlinear Optimization](https://mlsanity.com/paper/2207.09442/) (2022)
- [A Large-Scale Outdoor Multi-modal Dataset and Benchmark for Novel View Synthesis and Implicit Scene Reconstruction](https://mlsanity.com/paper/2301.06782/) (2023)
- [PyPose: A Library for Robot Learning with Physics-based Optimization](https://mlsanity.com/paper/2209.15428/) (2022)
- [gradSLAM: Automagically differentiable SLAM](https://mlsanity.com/paper/1910.10672/) (2019)
- [Self-supervised Geometric Perception](https://mlsanity.com/paper/2103.03114/) (2021)
- [AirSLAM: An Efficient and Illumination-Robust Point-Line Visual SLAM System](https://mlsanity.com/paper/2408.03520/) (2024)
- [TensorFlow Eager: A Multi-Stage, Python-Embedded DSL for Machine Learning](https://mlsanity.com/paper/1903.01855/) (2019)
- [Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy](https://mlsanity.com/paper/2406.16087/) (2024)
- [iSLAM: Imperative SLAM](https://mlsanity.com/paper/2306.07894/) (2023)
- [AirVO: An Illumination-Robust Point-Line Visual Odometry](https://mlsanity.com/paper/2212.07595/) (2022)
- [PyPose v0.6: The Imperative Programming Interface for Robotics](https://mlsanity.com/paper/2309.13035/) (2023)

## Built on by (most cited first, 6 of 6)

- [Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond](https://mlsanity.com/paper/2604.22748/) (2026)
- [InstantSfM: Towards GPU-Native SfM for the Deep Learning Era](https://mlsanity.com/paper/2510.13310/) (2025)
- [Glob3R: Global Structure-from-Motion with 3D Foundation Models](https://mlsanity.com/paper/2607.09225/) (2026)
- [GloSplat: Joint Pose-Appearance Optimization for Faster and More Accurate 3D Reconstruction](https://mlsanity.com/paper/2603.04847/) (2026)
- [DGSfM: Depth-Guided Scale-Aware Global Structure-from-Motion](https://mlsanity.com/paper/2607.09507/) (2026)
- [CSS-BA: Gate-Guided Column Space Search for Bundle Adjustment](https://mlsanity.com/paper/2607.15652/) (2026)

## Abstract (arXiv)

Bundle adjustment (BA) is a critical technique in various robotic applications such as simultaneous localization and mapping (SLAM), augmented reality (AR), and photogrammetry. BA optimizes parameters such as camera poses and 3D landmarks to align them with observations. With the growing importance of deep learning in perception systems, there is an increasing need to integrate BA with deep learning frameworks for enhanced reliability and performance. However, widely-used C++-based BA libraries, such as GTSAM, g2o, and Ceres Solver, lack native integration with modern deep learning libraries like PyTorch. This limitation affects their flexibility, ease of debugging, and overall implementation efficiency. To address this gap, we introduce an eager-mode BA library seamlessly integrated with PyTorch with high efficiency. Our approach includes a sparsity-aware auto-differentiation design and GPU-accelerated sparse operations designed for 2nd-order optimization. Our eager-mode BA on GPU demonstrates substantial runtime efficiency, achieving an average speedup of 18.5×, 22×, and 23× across all benchmarks compared to GTSAM, g2o, and Ceres, respectively.