# Next-Best View Policy for 3D Reconstruction

arXiv: https://arxiv.org/abs/2008.12664 · HTML: https://mlsanity.com/paper/2008.12664/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2020.

## Built on by (most cited first, 30 of 35)

- [ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis](https://mlsanity.com/paper/2409.02048/) (2024)
- [NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction with Implicit Neural Representations](https://mlsanity.com/paper/2207.10985/) (2022)
- [Uncertainty Guided Policy for Active Robotic 3D Reconstruction using Neural Radiance Fields](https://mlsanity.com/paper/2209.08409/) (2022)
- [RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation](https://mlsanity.com/paper/2402.15487/) (2024)
- [GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction](https://mlsanity.com/paper/2402.16174/) (2024)
- [ActiveRMAP: Radiance Field for Active Mapping And Planning](https://mlsanity.com/paper/2211.12656/) (2022)
- [NARUTO: Neural Active Reconstruction from Uncertain Target Observations](https://mlsanity.com/paper/2402.18771/) (2024)
- [SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Rendering](https://mlsanity.com/paper/2210.15185/) (2022)
- [ActiveGAMER: Active GAussian Mapping through Efficient Rendering](https://mlsanity.com/paper/2501.06897/) (2025)
- [A Review on Viewpoints and Path-planning for UAV-based 3D Reconstruction](https://mlsanity.com/paper/2205.03716/) (2022)
- [ShaDDR: Interactive Example-Based Geometry and Texture Generation via 3D Shape Detailization and Differentiable Rendering](https://mlsanity.com/paper/2306.04889/) (2023)
- [NextBestPath: Efficient 3D Mapping of Unseen Environments](https://mlsanity.com/paper/2502.05378/) (2025)
- [Attention-driven Next-best-view Planning for Efficient Reconstruction of Plants and Targeted Plant Parts](https://mlsanity.com/paper/2206.10274/) (2022)
- [Uncertainty-Driven Active Vision for Implicit Scene Reconstruction](https://mlsanity.com/paper/2210.00978/) (2022)
- [Frequency-based View Selection in Gaussian Splatting Reconstruction](https://mlsanity.com/paper/2409.16470/) (2024)
- [Bag of Views: An Appearance-based Approach to Next-Best-View Planning for 3D Reconstruction](https://mlsanity.com/paper/2307.05832/) (2023)
- [VIN-NBV: A View Introspection Network for Next-Best-View Selection](https://mlsanity.com/paper/2505.06219/) (2025)
- [ACT-R: Adaptive Camera Trajectories for Single View 3D Reconstruction](https://mlsanity.com/paper/2505.08239/) (2025)
- [ObjSplat: Geometry-Aware Gaussian Surfels for Active Object Reconstruction](https://mlsanity.com/paper/2601.06997/) (2026)
- [FArMARe: a Furniture-Aware Multi-task methodology for Recommending Apartments based on the user interests](https://mlsanity.com/paper/2309.03100/) (2023)
- [Scalable Multi-Robot Informative Path Planning for Target Mapping via Deep Reinforcement Learning](https://mlsanity.com/paper/2409.16967/) (2024)
- [Hestia: Voxel-Face-Aware Hierarchical Next-Best-View Acquisition for Efficient 3D Reconstruction](https://mlsanity.com/paper/2508.01014/) (2025)
- [CoRe-GS: Coarse-to-Refined Gaussian Splatting with Semantic Object Focus](https://mlsanity.com/paper/2509.04859/) (2025)
- [BuildingWorld: A Structured 3D Building Dataset for Urban Foundation Models](https://mlsanity.com/paper/2511.06337/) (2025)
- [Next-Best-View Estimation based on Deep Reinforcement Learning for Active Object Classification](https://mlsanity.com/paper/2110.06766/) (2021)
- [IOVS4NeRF:Incremental Optimal View Selection for Large-Scale NeRFs](https://mlsanity.com/paper/2407.18611/) (2024)
- [ArchComplete: Autoregressive 3D Architectural Design Generation with Hierarchical Diffusion-Based Upsampling](https://mlsanity.com/paper/2412.17957/) (2024)
- [Task-Driven Implicit Representations for Automated Design of LiDAR Systems](https://mlsanity.com/paper/2505.22344/) (2025)
- [Geometry-Aware Physics-Informed PointNets for Modeling Flows Across Porous Structures](https://mlsanity.com/paper/2602.14108/) (2026)
- [BayesFusion-SDF: Probabilistic Signed Distance Fusion with View Planning on CPU](https://mlsanity.com/paper/2602.19697/) (2026)

## Abstract (arXiv)

Manually selecting viewpoints or using commonly available flight planners like circular path for large-scale 3D reconstruction using drones often results in incomplete 3D models. Recent works have relied on hand-engineered heuristics such as information gain to select the Next-Best Views. In this work, we present a learning-based algorithm called Scan-RL to learn a Next-Best View (NBV) Policy. To train and evaluate the agent, we created Houses3K, a dataset of 3D house models. Our experiments show that using Scan-RL, the agent can scan houses with fewer number of steps and a shorter distance compared to our baseline circular path. Experimental results also demonstrate that a single NBV policy can be used to scan multiple houses including those that were not seen during training. The link to Scan-RL is available at https://github.com/darylperalta/ScanRL and Houses3K dataset can be found at https://github.com/darylperalta/Houses3K.