# NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction with Implicit Neural Representations

arXiv: https://arxiv.org/abs/2207.10985 · HTML: https://mlsanity.com/paper/2207.10985/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 22 Jul 2022.

## Builds on (most cited first, 19 of 19)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [Learning to Explore using Active Neural SLAM](https://mlsanity.com/paper/2004.05155/) (2020)
- [Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Images](https://mlsanity.com/paper/1901.11153/) (2019)
- [NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction](https://mlsanity.com/paper/2203.11283/) (2022)
- [Intel RealSense Stereoscopic Depth Cameras](https://mlsanity.com/paper/1705.05548/) (2017)
- [An Efficient Sampling-based Method for Online Informative Path Planning in Unknown Environments](https://mlsanity.com/paper/1909.09548/) (2019)
- [RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction](https://mlsanity.com/paper/2011.14744/) (2020)
- [FloorNet: A Unified Framework for Floorplan Reconstruction from 3D Scans](https://mlsanity.com/paper/1804.00090/) (2018)
- [Next-Best View Policy for 3D Reconstruction](https://mlsanity.com/paper/2008.12664/) (2020)
- [Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility](https://mlsanity.com/paper/1806.10354/) (2018)
- [Occlusion-aware Hand Pose Estimation Using Hierarchical Mixture Density Network](https://mlsanity.com/paper/1711.10872/) (2017)
- [A Reinforcement Learning Approach to the View Planning Problem](https://mlsanity.com/paper/1610.06204/) (2016)
- [Tackling Data Heterogeneity: A New Unified Framework for Decentralized SGD with Sample-induced Topology](https://mlsanity.com/paper/2207.03730/) (2022)
- [Fast and Compute-efficient Sampling-based Local Exploration Planning via Distribution Learning](https://mlsanity.com/paper/2202.13715/) (2022)

## Built on by (most cited first, 30 of 55)

- [Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields](https://mlsanity.com/paper/2309.03185/) (2023)
- [NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild](https://mlsanity.com/paper/2405.18715/) (2024)
- [FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information](https://mlsanity.com/paper/2311.17874/) (2023)
- [GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction](https://mlsanity.com/paper/2402.16174/) (2024)
- [ActiveGS: Active Scene Reconstruction Using Gaussian Splatting](https://mlsanity.com/paper/2412.17769/) (2024)
- [Active Neural Mapping](https://mlsanity.com/paper/2308.16246/) (2023)
- [ActiveRMAP: Radiance Field for Active Mapping And Planning](https://mlsanity.com/paper/2211.12656/) (2022)
- [Active Implicit Object Reconstruction using Uncertainty-guided Next-Best-View Optimization](https://mlsanity.com/paper/2303.16739/) (2023)
- [NARUTO: Neural Active Reconstruction from Uncertain Target Observations](https://mlsanity.com/paper/2402.18771/) (2024)
- [Neural Visibility Field for Uncertainty-Driven Active Mapping](https://mlsanity.com/paper/2406.06948/) (2024)
- [Improving NeRF Quality by Progressive Camera Placement for Unrestricted Navigation in Complex Environments](https://mlsanity.com/paper/2309.00014/) (2023)
- [ActiveGAMER: Active GAussian Mapping through Efficient Rendering](https://mlsanity.com/paper/2501.06897/) (2025)
- [Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields](https://mlsanity.com/paper/2307.15131/) (2023)
- [AtomGS: Atomizing Gaussian Splatting for High-Fidelity Radiance Field](https://mlsanity.com/paper/2405.12369/) (2024)
- [GauSS-MI: Gaussian Splatting Shannon Mutual Information for Active 3D Reconstruction](https://mlsanity.com/paper/2504.21067/) (2025)
- [Estimating 3D Uncertainty Field: Quantifying Uncertainty for Neural Radiance Fields](https://mlsanity.com/paper/2311.01815/) (2023)
- [NeRF Director: Revisiting View Selection in Neural Volume Rendering](https://mlsanity.com/paper/2406.08839/) (2024)
- [NextBestPath: Efficient 3D Mapping of Unseen Environments](https://mlsanity.com/paper/2502.05378/) (2025)
- [Neural Radiance Fields for the Real World: A Survey](https://mlsanity.com/paper/2501.13104/) (2025)
- [GLEAM: Learning Generalizable Exploration Policy for Active Mapping in Complex 3D Indoor Scenes](https://mlsanity.com/paper/2505.20294/) (2025)
- [Sources of Uncertainty in 3D Scene Reconstruction](https://mlsanity.com/paper/2409.06407/) (2024)
- [DGNR: Density-Guided Neural Point Rendering of Large Driving Scenes](https://mlsanity.com/paper/2311.16664/) (2023)
- [Active Neural Mapping at Scale](https://mlsanity.com/paper/2409.20276/) (2024)
- [SO-NeRF: Active View Planning for NeRF using Surrogate Objectives](https://mlsanity.com/paper/2312.03266/) (2023)
- [PC-NeRF: Parent-Child Neural Radiance Fields Using Sparse LiDAR Frames in Autonomous Driving Environments](https://mlsanity.com/paper/2402.09325/) (2024)
- [Understanding while Exploring: Semantics-driven Active Mapping](https://mlsanity.com/paper/2506.00225/) (2025)
- [Peering into the Unknown: Active View Selection with Neural Uncertainty Maps for 3D Reconstruction](https://mlsanity.com/paper/2506.14856/) (2025)
- [Molecular Representations in Implicit Functional Space via Hyper-Networks](https://mlsanity.com/paper/2601.22327/) (2026)
- [Uncertainty-driven 3D Gaussian Splatting Active Mapping via Anisotropic Visibility Field](https://mlsanity.com/paper/2605.30342/) (2026)
- [Auto3R: Automated 3D Reconstruction and Scanning via Data-driven Uncertainty Quantification](https://mlsanity.com/paper/2512.04528/) (2025)

## Abstract (arXiv)

Implicit neural representations have shown compelling results in offline 3D reconstruction and also recently demonstrated the potential for online SLAM systems. However, applying them to autonomous 3D reconstruction, where a robot is required to explore a scene and plan a view path for the reconstruction, has not been studied. In this paper, we explore for the first time the possibility of using implicit neural representations for autonomous 3D scene reconstruction by addressing two key challenges: 1) seeking a criterion to measure the quality of the candidate viewpoints for the view planning based on the new representations, and 2) learning the criterion from data that can generalize to different scenes instead of a hand-crafting one. To solve the challenges, firstly, a proxy of Peak Signal-to-Noise Ratio (PSNR) is proposed to quantify a viewpoint quality; secondly, the proxy is optimized jointly with the parameters of an implicit neural network for the scene. With the proposed view quality criterion from neural networks (termed as Neural Uncertainty), we can then apply implicit representations to autonomous 3D reconstruction. Our method demonstrates significant improvements on various metrics for the rendered image quality and the geometry quality of the reconstructed 3D models when compared with variants using TSDF or reconstruction without view planning. Project webpage https://kingteeloki-ran.github.io/NeurAR/