# ActiveNeRF: Learning where to See with Uncertainty Estimation

arXiv: https://arxiv.org/abs/2209.08546 · HTML: https://mlsanity.com/paper/2209.08546/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 37 references that are in the map.
- First posted on arXiv 18 Sep 2022.

## Builds on (most cited first, 21 of 21)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis](https://mlsanity.com/paper/2104.00677/) (2021)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [GRF: Learning a General Radiance Field for 3D Representation and Rendering](https://mlsanity.com/paper/2010.04595/) (2020)
- [NeRF-VAE: A Geometry Aware 3D Scene Generative Model](https://mlsanity.com/paper/2104.00587/) (2021)
- [Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representations](https://mlsanity.com/paper/2109.02123/) (2021)
- [Reducing Label Effort: Self-Supervised meets Active Learning](https://mlsanity.com/paper/2108.11458/) (2021)
- [NeRF in detail: Learning to sample for view synthesis](https://mlsanity.com/paper/2106.05264/) (2021)
- [Temporal Coherence for Active Learning in Videos](https://mlsanity.com/paper/1908.11757/) (2019)

## Built on by (most cited first, 30 of 127)

- [3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors](https://mlsanity.com/paper/2410.16266/) (2024)
- [Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting](https://mlsanity.com/paper/2404.03613/) (2024)
- [PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting](https://mlsanity.com/paper/2406.10219/) (2024)
- [Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields](https://mlsanity.com/paper/2309.03185/) (2023)
- [NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild](https://mlsanity.com/paper/2405.18715/) (2024)
- [Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection](https://mlsanity.com/paper/2303.10840/) (2023)
- [FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information](https://mlsanity.com/paper/2311.17874/) (2023)
- [PhyRecon: Physically Plausible Neural Scene Reconstruction](https://mlsanity.com/paper/2404.16666/) (2024)
- [GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction](https://mlsanity.com/paper/2402.16174/) (2024)
- [ActiveGS: Active Scene Reconstruction Using Gaussian Splatting](https://mlsanity.com/paper/2412.17769/) (2024)
- [ActiveSplat: High-Fidelity Scene Reconstruction through Active Gaussian Splatting](https://mlsanity.com/paper/2410.21955/) (2024)
- [Active Neural Mapping](https://mlsanity.com/paper/2308.16246/) (2023)
- [ActiveRMAP: Radiance Field for Active Mapping And Planning](https://mlsanity.com/paper/2211.12656/) (2022)
- [What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Models](https://mlsanity.com/paper/2512.03422/) (2025)
- [Active Implicit Object Reconstruction using Uncertainty-guided Next-Best-View Optimization](https://mlsanity.com/paper/2303.16739/) (2023)
- [NARUTO: Neural Active Reconstruction from Uncertain Target Observations](https://mlsanity.com/paper/2402.18771/) (2024)
- [4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization](https://mlsanity.com/paper/2411.08879/) (2024)
- [FlowR: Flowing from Sparse to Dense 3D Reconstructions](https://mlsanity.com/paper/2504.01647/) (2025)
- [Neural Visibility Field for Uncertainty-Driven Active Mapping](https://mlsanity.com/paper/2406.06948/) (2024)
- [Improving NeRF Quality by Progressive Camera Placement for Unrestricted Navigation in Complex Environments](https://mlsanity.com/paper/2309.00014/) (2023)
- [ActiveGAMER: Active GAussian Mapping through Efficient Rendering](https://mlsanity.com/paper/2501.06897/) (2025)
- [POp-GS: Next Best View in 3D-Gaussian Splatting with P-Optimality](https://mlsanity.com/paper/2503.07819/) (2025)
- [iVR-GS: Inverse Volume Rendering for Explorable Visualization via Editable 3D Gaussian Splatting](https://mlsanity.com/paper/2504.17954/) (2025)
- [DebSDF: Delving into the Details and Bias of Neural Indoor Scene Reconstruction](https://mlsanity.com/paper/2308.15536/) (2023)
- [GauSS-MI: Gaussian Splatting Shannon Mutual Information for Active 3D Reconstruction](https://mlsanity.com/paper/2504.21067/) (2025)
- [Estimating 3D Uncertainty Field: Quantifying Uncertainty for Neural Radiance Fields](https://mlsanity.com/paper/2311.01815/) (2023)
- [NeRF Director: Revisiting View Selection in Neural Volume Rendering](https://mlsanity.com/paper/2406.08839/) (2024)
- [NextBestPath: Efficient 3D Mapping of Unseen Environments](https://mlsanity.com/paper/2502.05378/) (2025)
- [Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization](https://mlsanity.com/paper/2310.06984/) (2023)
- [Manifold Sampling for Differentiable Uncertainty in Radiance Fields](https://mlsanity.com/paper/2409.12661/) (2024)

## Abstract (arXiv)

Recently, Neural Radiance Fields (NeRF) has shown promising performances on reconstructing 3D scenes and synthesizing novel views from a sparse set of 2D images. Albeit effective, the performance of NeRF is highly influenced by the quality of training samples. With limited posed images from the scene, NeRF fails to generalize well to novel views and may collapse to trivial solutions in unobserved regions. This makes NeRF impractical under resource-constrained scenarios. In this paper, we present a novel learning framework, ActiveNeRF, aiming to model a 3D scene with a constrained input budget. Specifically, we first incorporate uncertainty estimation into a NeRF model, which ensures robustness under few observations and provides an interpretation of how NeRF understands the scene. On this basis, we propose to supplement the existing training set with newly captured samples based on an active learning scheme. By evaluating the reduction of uncertainty given new inputs, we select the samples that bring the most information gain. In this way, the quality of novel view synthesis can be improved with minimal additional resources. Extensive experiments validate the performance of our model on both realistic and synthetic scenes, especially with scarcer training data. Code will be released at urlhttps://github.com/LeapLabTHU/ActiveNeRF.