# VIN-NBV: A View Introspection Network for Next-Best-View Selection

arXiv: https://arxiv.org/abs/2505.06219 · HTML: https://mlsanity.com/paper/2505.06219/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 51 references that are in the map.
- First posted on arXiv 9 May 2025.

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation](https://mlsanity.com/paper/2312.02145/) (2023)
- [OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation](https://mlsanity.com/paper/2301.07525/) (2023)
- [Rank consistent ordinal regression for neural networks with application to age estimation](https://mlsanity.com/paper/1901.07884/) (2019)
- [ActiveNeRF: Learning where to See with Uncertainty Estimation](https://mlsanity.com/paper/2209.08546/) (2022)
- [FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information](https://mlsanity.com/paper/2311.17874/) (2023)
- [NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction with Implicit Neural Representations](https://mlsanity.com/paper/2207.10985/) (2022)
- [Uncertainty Guided Policy for Active Robotic 3D Reconstruction using Neural Radiance Fields](https://mlsanity.com/paper/2209.08409/) (2022)
- [MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo](https://mlsanity.com/paper/2401.11673/) (2024)
- [GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction](https://mlsanity.com/paper/2402.16174/) (2024)
- [Next-Best View Policy for 3D Reconstruction](https://mlsanity.com/paper/2008.12664/) (2020)
- [MACARONS: Mapping And Coverage Anticipation with RGB Online Self-Supervision](https://mlsanity.com/paper/2303.03315/) (2023)
- [SCONE: Surface Coverage Optimization in Unknown Environments by Volumetric Integration](https://mlsanity.com/paper/2208.10449/) (2022)
- [Submodular Trajectory Optimization for Aerial 3D Scanning](https://mlsanity.com/paper/1705.00703/) (2017)
- [Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility](https://mlsanity.com/paper/1806.10354/) (2018)
- [A Reinforcement Learning Approach to the View Planning Problem](https://mlsanity.com/paper/1610.06204/) (2016)
- [Uncertainty-Driven Active Vision for Implicit Scene Reconstruction](https://mlsanity.com/paper/2210.00978/) (2022)
- [SO-NeRF: Active View Planning for NeRF using Surrogate Objectives](https://mlsanity.com/paper/2312.03266/) (2023)

## Built on by (most cited first, 2 of 2)

- [Scene-SAM3D: Multi-View Scene Asset Generation Without Fine-Tuning](https://mlsanity.com/paper/2607.16805/) (2026)
- [DA-NBV: A Direction-Aware Next-Best-View Planner for Efficient 3D Reconstruction of Ships at Sea](https://mlsanity.com/paper/2608.08025/) (2026)

## Abstract (arXiv)

Next Best View (NBV) algorithms aim to maximize 3D scene acquisition quality using minimal resources, e.g. number of acquisitions, time taken, or distance traversed. Prior methods often rely on coverage maximization as a proxy for reconstruction quality, but for complex scenes with occlusions and finer details, this is not always sufficient and leads to poor reconstructions. Our key insight is to train an acquisition policy that directly optimizes for reconstruction quality rather than just coverage. To achieve this, we introduce the View Introspection Network (VIN): a lightweight neural network that predicts the Relative Reconstruction Improvement (RRI) of a potential next viewpoint without making any new acquisitions. We use this network to power a simple, yet effective, sequential samplingbased greedy NBV policy. Our approach, VIN-NBV, generalizes to unseen object categories, operates without prior scene knowledge, is adaptable to resource constraints, and can handle occlusions. We show that our RRI fitness criterion leads to a ~30% gain in reconstruction quality over a coverage-based criterion using the same greedy strategy. Furthermore, VIN-NBV also outperforms deep reinforcement learning methods, Scan-RL and GenNBV, by ~40%.