# Robust Incremental Structure-from-Motion with Hybrid Features

arXiv: https://arxiv.org/abs/2409.19811 · HTML: https://mlsanity.com/paper/2409.19811/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 115 references that are in the map.
- First posted on arXiv 29 Sep 2024.

## Builds on (most cited first, 30 of 42)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction](https://mlsanity.com/paper/2206.00665/) (2022)
- [Neuralangelo: High-Fidelity Neural Surface Reconstruction](https://mlsanity.com/paper/2306.03092/) (2023)
- [Visual Geometry Grounded Deep Structure From Motion](https://mlsanity.com/paper/2312.04563/) (2023)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [BA-Net: Dense Bundle Adjustment Network](https://mlsanity.com/paper/1806.04807/) (2018)
- [PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment](https://mlsanity.com/paper/2306.15667/) (2023)
- [Cameras as Rays: Pose Estimation via Ray Diffusion](https://mlsanity.com/paper/2402.14817/) (2024)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [Detector-Free Structure from Motion](https://mlsanity.com/paper/2306.15669/) (2023)
- [GlueStick: Robust Image Matching by Sticking Points and Lines Together](https://mlsanity.com/paper/2304.02008/) (2023)
- [Holistically-Attracted Wireframe Parsing](https://mlsanity.com/paper/2003.01663/) (2020)
- [DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients](https://mlsanity.com/paper/2212.07766/) (2022)
- [PL-SLAM: a Stereo SLAM System through the Combination of Points and Line Segments](https://mlsanity.com/paper/1705.09479/) (2017)
- [SOLD2: Self-supervised Occlusion-aware Line Description and Detection](https://mlsanity.com/paper/2104.03362/) (2021)
- [3D Line Mapping Revisited](https://mlsanity.com/paper/2303.17504/) (2023)
- [TP-LSD: Tri-Points Based Line Segment Detector](https://mlsanity.com/paper/2009.05505/) (2020)
- [Multi-View Optimization of Local Feature Geometry](https://mlsanity.com/paper/2003.08348/) (2020)
- [Robust Visual SLAM with Point and Line Features](https://mlsanity.com/paper/1711.08654/) (2017)
- [DeepC-MVS: Deep Confidence Prediction for Multi-View Stereo Reconstruction](https://mlsanity.com/paper/1912.00439/) (2019)
- [S2DNet: Learning Accurate Correspondences for Sparse-to-Dense Feature Matching](https://mlsanity.com/paper/2004.01673/) (2020)
- [UV-SLAM: Unconstrained Line-based SLAM Using Vanishing Points for Structural Mapping](https://mlsanity.com/paper/2112.13515/) (2021)

## Built on by (most cited first, 30 of 31)

- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Streaming 4D Visual Geometry Transformer](https://mlsanity.com/paper/2507.11539/) (2025)
- [Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization](https://mlsanity.com/paper/2412.08376/) (2024)
- [InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streams](https://mlsanity.com/paper/2601.02281/) (2026)
- [VGGT-Omega](https://mlsanity.com/paper/2605.15195/) (2026)
- [ViSTA-SLAM: Visual SLAM with Symmetric Two-view Association](https://mlsanity.com/paper/2509.01584/) (2025)
- [TTSA3R: Training-Free Temporal-Spatial Adaptive Persistent State for Streaming 3D Reconstruction](https://mlsanity.com/paper/2601.22615/) (2026)
- [4D-VGGT: A General Foundation Model with SpatioTemporal Awareness for Dynamic Scene Geometry Estimation](https://mlsanity.com/paper/2511.18416/) (2025)
- [Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learning](https://mlsanity.com/paper/2602.20157/) (2026)
- [Global Structure-from-Motion Meets Feedforward Reconstruction](https://mlsanity.com/paper/2605.26103/) (2026)
- [Vision Pretraining for Dense Spatial Perception](https://mlsanity.com/paper/2607.05247/) (2026)
- [TALO: Pushing 3D Vision Foundation Models Towards Globally Consistent Online Reconstruction](https://mlsanity.com/paper/2512.02341/) (2025)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [DriveVGGT: Calibration-Constrained Visual Geometry Transformers for Multi-Camera Autonomous Driving](https://mlsanity.com/paper/2511.22264/) (2025)
- [Attention Itself Could Retrieve.RetrieveVGGT: Training-Free Long Context Streaming 3D Reconstruction via Query-Key Similarity Retrieval](https://mlsanity.com/paper/2605.09644/) (2026)
- [A Taxonomy of Structure from Motion Methods](https://mlsanity.com/paper/2505.15814/) (2025)
- [CVD-SfM: A Cross-View Deep Front-end Structure-from-Motion System for Sparse Localization in Multi-Altitude Scenes](https://mlsanity.com/paper/2508.01936/) (2025)
- [LightGlueStick: a Fast and Robust Glue for Joint Point-Line Matching](https://mlsanity.com/paper/2510.16438/) (2025)
- [Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence](https://mlsanity.com/paper/2605.19727/) (2026)
- [VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion](https://mlsanity.com/paper/2607.27194/) (2026)
- [PAOLI: Pose-free Articulated Object Learning from Sparse-view Images](https://mlsanity.com/paper/2509.04276/) (2025)
- [From Fields to Splats: A Cross-Domain Survey of Real-Time Neural Scene Representations](https://mlsanity.com/paper/2509.23555/) (2025)
- [AHAP: Reconstructing Arbitrary Humans from Arbitrary Perspectives with Geometric Priors](https://mlsanity.com/paper/2602.23951/) (2026)
- [QVGGT: Post-Training Quantized Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2605.31124/) (2026)
- [UniSim-SLAM: Feed-Forward SLAM with Unified Sim(3) Optimization](https://mlsanity.com/paper/2608.01706/) (2026)
- [T-Graph: Enhancing Sparse-view Camera Pose Estimation by Pairwise Translation Graph](https://mlsanity.com/paper/2505.01207/) (2025)
- [BA-T: An Iterative Transformer for Two-View Bundle Adjustment](https://mlsanity.com/paper/2606.03287/) (2026)
- [EPO: Boosting 3D Foundation Models with Edge-based Pose Optimization](https://mlsanity.com/paper/2607.00579/) (2026)
- [Unified and Efficient Point-Line Local Features](https://mlsanity.com/paper/2608.19894/) (2026)
- [Stable and Scalable Bundle Adjustment of Holistic 3D Structures](https://mlsanity.com/paper/2609.04026/) (2026)

## Abstract (arXiv)

Structure-from-Motion (SfM) has become a ubiquitous tool for camera calibration and scene reconstruction with many downstream applications in computer vision and beyond. While the state-of-the-art SfM pipelines have reached a high level of maturity in well-textured and well-configured scenes over the last decades, they still fall short of robustly solving the SfM problem in challenging scenarios. In particular, weakly textured scenes and poorly constrained configurations oftentimes cause catastrophic failures or large errors for the primarily keypoint-based pipelines. In these scenarios, line segments are often abundant and can offer complementary geometric constraints. Their large spatial extent and typically structured configurations lead to stronger geometric constraints as compared to traditional keypoint-based methods. In this work, we introduce an incremental SfM system that, in addition to points, leverages lines and their structured geometric relations. Our technical contributions span the entire pipeline (mapping, triangulation, registration) and we integrate these into a comprehensive end-to-end SfM system that we share as an open-source software with the community. We also present the first analytical method to propagate uncertainties for 3D optimized lines via sensitivity analysis. Experiments show that our system is consistently more robust and accurate compared to the widely used point-based state of the art in SfM -- achieving richer maps and more precise camera registrations, especially under challenging conditions. In addition, our uncertainty-aware localization module alone is able to consistently improve over the state of the art under both point-alone and hybrid setups.