# SfM on-the-fly: Get better 3D from What You Capture

arXiv: https://arxiv.org/abs/2407.03939 · HTML: https://mlsanity.com/paper/2407.03939/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 11 references that are in the map.
- First posted on arXiv 4 Jul 2024.

## Builds on (most cited first, 5 of 5)

- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Image Retrieval for Structure-from-Motion via Graph Convolutional Network](https://mlsanity.com/paper/2009.08049/) (2020)
- [On-the-Fly SfM: What you capture is What you get](https://mlsanity.com/paper/2309.11883/) (2023)
- [OpenREALM: Real-time Mapping for Unmanned Aerial Vehicles](https://mlsanity.com/paper/2009.10492/) (2020)

## Built on by (most cited first, 2 of 2)

- [Gaussian On-the-Fly Splatting: A Progressive Framework for Robust Near Real-Time 3DGS Optimization](https://mlsanity.com/paper/2503.13086/) (2025)
- [A-TDOM: Active TDOM via On-the-Fly 3DGS](https://mlsanity.com/paper/2509.12759/) (2025)

## Abstract (arXiv)

In the last twenty years, Structure from Motion (SfM) has been a constant research hotspot in the fields of photogrammetry, computer vision, robotics etc., whereas real-time performance is just a recent topic of growing interest. This work builds upon the original on-the-fly SfM (Zhan et al., 2024) and presents an updated version with three new advancements to get better 3D from what you capture: (i) real-time image matching is further boosted by employing the Hierarchical Navigable Small World (HNSW) graphs, thus more true positive overlapping image candidates are faster identified; (ii) a self-adaptive weighting strategy is proposed for robust hierarchical local bundle adjustment to improve the SfM results; (iii) multiple agents are included for supporting collaborative SfM and seamlessly merge multiple 3D reconstructions into a complete 3D scene when commonly registered images appear. Various comprehensive experiments demonstrate that the proposed SfM method (named on-the-fly SfMv2) can generate more complete and robust 3D reconstructions in a high time-efficient way. Code is available at http://yifeiyu225.github.io/on-the-flySfMv2.github.io/.