Density Uncertainty Quantification with NeRF-Ensembles: Impact of Data and Scene Constraints
Not on the map. Why?
Abstract (arXiv)
In the fields of computer graphics, computer vision and photogrammetry, Neural Radiance Fields (NeRFs) are a major topic driving current research and development. However, the quality of NeRF-generated 3D scene reconstructions and subsequent surface reconstructions, heavily relies on the network output, particularly the density. Regarding this critical aspect, we propose to utilize NeRF-Ensembles that provide a density uncertainty estimate alongside the mean density. We demonstrate that data constraints such as low-quality images and poses lead to a degradation of the training process, increased density uncertainty and decreased predicted density. Even with high-quality input data, the density uncertainty varies based on scene constraints such as acquisition constellations, occlusions and material properties. NeRF-Ensembles not only provide a tool for quantifying the uncertainty but exhibit two promising advantages: Enhanced robustness and artifact removal. Through the utilization of NeRF-Ensembles instead of single NeRFs, small outliers are removed, yielding a smoother output with improved completeness of structures. Furthermore, applying percentile-based thresholds on density uncertainty outliers proves to be effective for the removal of large (foggy) artifacts in post-processing. We conduct our methodology on 3 different datasets: (i) synthetic benchmark dataset, (ii) real benchmark dataset, (iii) real data under realistic recording conditions and sensors.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 51 references that are in the map. First posted on arXiv 22 Dec 2023.
Builds on
34 of 51 references.
References · 34
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
- 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
- 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
- 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
- 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
22 more
- 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
- 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 464
- 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
- 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 336
- 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235
- 2021GNeRF: GAN-based Neural Radiance Field without Posed Cameracited by 153
- 2022ActiveNeRF: Learning where to See with Uncertainty Estimationcited by 124
- 2019A General Framework for Uncertainty Estimation in Deep Learningcited by 119
- 2023Bayes' Rays: Uncertainty Quantification for Neural Radiance Fieldscited by 90
- 2022GARF: Gaussian Activated Radiance Fields for High Fidelity Reconstruction and Pose Estimationcited by 86
- 2021Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representationscited by 80
- 2022Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantificationcited by 71
- 2022AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fieldscited by 68
- 2022Density-aware NeRF Ensembles: Quantifying Predictive Uncertainty in Neural Radiance Fieldscited by 58
- 2022Parallel Inversion of Neural Radiance Fields for Robust Pose Estimationcited by 55
- 2023ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fieldscited by 43
- 2023DUDES: Deep Uncertainty Distillation using Ensembles for Semantic Segmentationcited by 15
- 2023CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields from Imperfect Camera Posescited by 11
- 2022Learning Structured Gaussians to Approximate Deep Ensemblescited by 8
- 2023Clean-NeRF: Reformulating NeRF to account for View-Dependent Observationscited by 6
- 20233D Density-Gradient based Edge Detection on Neural Radiance Fields (NeRFs) for Geometric Reconstructioncited by 4
- 2023A Comparative Neural Radiance Field (NeRF) 3D Analysis of Camera Poses from HoloLens Trajectories and Structure from Motioncited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.