# Uncertainty-Driven Active Vision for Implicit Scene Reconstruction

arXiv: https://arxiv.org/abs/2210.00978 · HTML: https://mlsanity.com/paper/2210.00978/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 3 Oct 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [ABC: A Big CAD Model Dataset For Geometric Deep Learning](https://mlsanity.com/paper/1812.06216/) (2018)
- [Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Images](https://mlsanity.com/paper/1901.11153/) (2019)
- [IoU Loss for 2D/3D Object Detection](https://mlsanity.com/paper/1908.03851/) (2019)
- [Next-Best View Policy for 3D Reconstruction](https://mlsanity.com/paper/2008.12664/) (2020)
- [Active 3D Shape Reconstruction from Vision and Touch](https://mlsanity.com/paper/2107.09584/) (2021)
- [Supervised Learning of the Next-Best-View for 3D Object Reconstruction](https://mlsanity.com/paper/1905.05833/) (2019)
- [Active Object Reconstruction Using a Guided View Planner](https://mlsanity.com/paper/1805.03081/) (2018)
- [Next-best-view Regression using a 3D Convolutional Neural Network](https://mlsanity.com/paper/2101.09397/) (2021)

## Built on by (most cited first, 6 of 6)

- [GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction](https://mlsanity.com/paper/2402.16174/) (2024)
- [Neural Visibility Field for Uncertainty-Driven Active Mapping](https://mlsanity.com/paper/2406.06948/) (2024)
- [Neural Stochastic Screened Poisson Reconstruction](https://mlsanity.com/paper/2309.11993/) (2023)
- [Peering into the Unknown: Active View Selection with Neural Uncertainty Maps for 3D Reconstruction](https://mlsanity.com/paper/2506.14856/) (2025)
- [VIN-NBV: A View Introspection Network for Next-Best-View Selection](https://mlsanity.com/paper/2505.06219/) (2025)
- [DAV-GSWT: Diffusion-Active-View Sampling for Data-Efficient Gaussian Splatting Wang Tiles](https://mlsanity.com/paper/2602.15355/) (2026)

## Abstract (arXiv)

Multi-view implicit scene reconstruction methods have become increasingly popular due to their ability to represent complex scene details. Recent efforts have been devoted to improving the representation of input information and to reducing the number of views required to obtain high quality reconstructions. Yet, perhaps surprisingly, the study of which views to select to maximally improve scene understanding remains largely unexplored. We propose an uncertainty-driven active vision approach for implicit scene reconstruction, which leverages occupancy uncertainty accumulated across the scene using volume rendering to select the next view to acquire. To this end, we develop an occupancy-based reconstruction method which accurately represents scenes using either 2D or 3D supervision. We evaluate our proposed approach on the ABC dataset and the in the wild CO3D dataset, and show that: (1) we are able to obtain high quality state-of-the-art occupancy reconstructions; (2) our perspective conditioned uncertainty definition is effective to drive improvements in next best view selection and outperforms strong baseline approaches; and (3) we can further improve shape understanding by performing a gradient-based search on the view selection candidates. Overall, our results highlight the importance of view selection for implicit scene reconstruction, making it a promising avenue to explore further.