# Learning Reconstructability for Drone Aerial Path Planning

arXiv: https://arxiv.org/abs/2209.10174 · HTML: https://mlsanity.com/paper/2209.10174/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [The ApolloScape Open Dataset for Autonomous Driving and its Application](https://mlsanity.com/paper/1803.06184/) (2018)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)
- [Capturing, Reconstructing, and Simulating: the UrbanScene3D Dataset](https://mlsanity.com/paper/2107.04286/) (2021)
- [TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers](https://mlsanity.com/paper/2111.14600/) (2021)
- [An Efficient Sampling-based Method for Online Informative Path Planning in Unknown Environments](https://mlsanity.com/paper/1909.09548/) (2019)
- [Submodular Trajectory Optimization for Aerial 3D Scanning](https://mlsanity.com/paper/1705.00703/) (2017)
- [Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility](https://mlsanity.com/paper/1806.10354/) (2018)
- [Plan3D: Viewpoint and Trajectory Optimization for Aerial Multi-View Stereo Reconstruction](https://mlsanity.com/paper/1705.09314/) (2017)
- [Adaptive View Planning for Aerial 3D Reconstruction](https://mlsanity.com/paper/1805.00506/) (2018)
- [VGF-Net: Visual-Geometric Fusion Learning for Simultaneous Drone Navigation and Height Mapping](https://mlsanity.com/paper/2104.03109/) (2021)
- [Learning Where to Look: Data-Driven Viewpoint Set Selection for 3D Scenes](https://mlsanity.com/paper/1704.02393/) (2017)

## Built on by (most cited first, 9 of 9)

- [GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction](https://mlsanity.com/paper/2402.16174/) (2024)
- [Optimizing Camera Configurations for Multi-View Pedestrian Detection](https://mlsanity.com/paper/2312.02144/) (2023)
- [ACT-R: Adaptive Camera Trajectories for Single View 3D Reconstruction](https://mlsanity.com/paper/2505.08239/) (2025)
- [Aerial Path Planning for Urban Geometry and Texture Co-Capture](https://mlsanity.com/paper/2509.22227/) (2025)
- [Active View Selection for Scene-level Multi-view Crowd Counting and Localization with Limited Labeling Budget](https://mlsanity.com/paper/2509.16684/) (2025)
- [FluidGaussian: Propagating Simulation-Based Uncertainty Toward Functionally-Intelligent 3D Reconstruction](https://mlsanity.com/paper/2603.21356/) (2026)
- [SceneFrom3D: Geometry-Conditioned Outdoor 3D Scene Generation via View Scheduling with Object-Level Control](https://mlsanity.com/paper/2607.04540/) (2026)
- [SiZeUp: Fast 3D Proxy from Aerial Images via Depth Ordinal Loss](https://mlsanity.com/paper/2608.22821/) (2026)
- [OpenFlyScan: A Quality-Guided Aerial Reconstruction System for Consumer Drones](https://mlsanity.com/paper/2609.24253/) (2026)

## Abstract (arXiv)

We introduce the first learning-based reconstructability predictor to improve view and path planning for large-scale 3D urban scene acquisition using unmanned drones. In contrast to previous heuristic approaches, our method learns a model that explicitly predicts how well a 3D urban scene will be reconstructed from a set of viewpoints. To make such a model trainable and simultaneously applicable to drone path planning, we simulate the proxy-based 3D scene reconstruction during training to set up the prediction. Specifically, the neural network we design is trained to predict the scene reconstructability as a function of the proxy geometry, a set of viewpoints, and optionally a series of scene images acquired in flight. To reconstruct a new urban scene, we first build the 3D scene proxy, then rely on the predicted reconstruction quality and uncertainty measures by our network, based off of the proxy geometry, to guide the drone path planning. We demonstrate that our data-driven reconstructability predictions are more closely correlated to the true reconstruction quality than prior heuristic measures. Further, our learned predictor can be easily integrated into existing path planners to yield improvements. Finally, we devise a new iterative view planning framework, based on the learned reconstructability, and show superior performance of the new planner when reconstructing both synthetic and real scenes.