# Bag of Views: An Appearance-based Approach to Next-Best-View Planning for 3D Reconstruction

arXiv: https://arxiv.org/abs/2307.05832 · HTML: https://mlsanity.com/paper/2307.05832/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 42 references that are in the map.
- First posted on arXiv 11 Jul 2023.

## Builds on (most cited first, 12 of 12)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1805.01956/) (2018)
- [Next-Best View Policy for 3D Reconstruction](https://mlsanity.com/paper/2008.12664/) (2020)
- [Where to Look Next: Learning Viewpoint Recommendations for Informative Trajectory Planning](https://mlsanity.com/paper/2203.02381/) (2022)
- [A Review on Viewpoints and Path-planning for UAV-based 3D Reconstruction](https://mlsanity.com/paper/2205.03716/) (2022)
- [BlendTorch: A Real-Time, Adaptive Domain Randomization Library](https://mlsanity.com/paper/2010.11696/) (2020)
- [PredRecon: A Prediction-boosted Planning Framework for Fast and High-quality Autonomous Aerial Reconstruction](https://mlsanity.com/paper/2302.04488/) (2023)

## Built on by (most cited first, 2 of 2)

- [On-the-fly Feedback SfM: Online Explore-and-Exploit UAV Photogrammetry with Incremental Mesh Quality-Aware Indicator and Predictive Path Planning](https://mlsanity.com/paper/2512.02375/) (2025)
- [Boundary Exploration of Next Best View Policy in 3D Robotic Scanning](https://mlsanity.com/paper/2412.10444/) (2024)

## Abstract (arXiv)

UAV-based intelligent data acquisition for 3D reconstruction and monitoring of infrastructure has experienced an increasing surge of interest due to recent advancements in image processing and deep learning-based techniques. View planning is an essential part of this task that dictates the information capture strategy and heavily impacts the quality of the 3D model generated from the captured data. Recent methods have used prior knowledge or partial reconstruction of the target to accomplish view planning for active reconstruction; the former approach poses a challenge for complex or newly identified targets while the latter is computationally expensive. In this work, we present Bag-of-Views (BoV), a fully appearance-based model used to assign utility to the captured views for both offline dataset refinement and online next-best-view (NBV) planning applications targeting the task of 3D reconstruction. With this contribution, we also developed the View Planning Toolbox (VPT), a lightweight package for training and testing machine learning-based view planning frameworks, custom view dataset generation of arbitrary 3D scenes, and 3D reconstruction. Through experiments which pair a BoV-based reinforcement learning model with VPT, we demonstrate the efficacy of our model in reducing the number of required views for high-quality reconstructions in dataset refinement and NBV planning.