# Manifold Sampling for Differentiable Uncertainty in Radiance Fields

arXiv: https://arxiv.org/abs/2409.12661 · HTML: https://mlsanity.com/paper/2409.12661/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 57 references that are in the map.
- First posted on arXiv 19 Sep 2024.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks](https://mlsanity.com/paper/2002.10118/) (2020)
- [DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Model](https://mlsanity.com/paper/2311.09217/) (2023)
- [DiffRF: Rendering-Guided 3D Radiance Field Diffusion](https://mlsanity.com/paper/2212.01206/) (2022)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam](https://mlsanity.com/paper/1806.04854/) (2018)
- [ActiveNeRF: Learning where to See with Uncertainty Estimation](https://mlsanity.com/paper/2209.08546/) (2022)
- [Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision](https://mlsanity.com/paper/2306.11719/) (2023)
- [Noisy Natural Gradient as Variational Inference](https://mlsanity.com/paper/1712.02390/) (2017)
- ['In-Between' Uncertainty in Bayesian Neural Networks](https://mlsanity.com/paper/1906.11537/) (2019)
- [Neural Light Transport for Relighting and View Synthesis](https://mlsanity.com/paper/2008.03806/) (2020)
- [Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantification](https://mlsanity.com/paper/2203.10192/) (2022)
- [FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information](https://mlsanity.com/paper/2311.17874/) (2023)
- [Neural Radiance Transfer Fields for Relightable Novel-view Synthesis with Global Illumination](https://mlsanity.com/paper/2207.13607/) (2022)
- [Neural Microfacet Fields for Inverse Rendering](https://mlsanity.com/paper/2303.17806/) (2023)

## Built on by (most cited first, 12 of 12)

- [PRIMU: Uncertainty Estimation for Novel Views in Gaussian Splatting from Primitive-Based Representations of Error and Coverage](https://mlsanity.com/paper/2508.02443/) (2025)
- [Predictive Photometric Uncertainty in Gaussian Splatting for Novel View Synthesis](https://mlsanity.com/paper/2603.22786/) (2026)
- [Uncertainty-driven 3D Gaussian Splatting Active Mapping via Anisotropic Visibility Field](https://mlsanity.com/paper/2605.30342/) (2026)
- [Auto3R: Automated 3D Reconstruction and Scanning via Data-driven Uncertainty Quantification](https://mlsanity.com/paper/2512.04528/) (2025)
- [WarpRF: Multi-View Consistency for Training-Free Uncertainty Quantification and Applications in Radiance Fields](https://mlsanity.com/paper/2506.22433/) (2025)
- [SA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection](https://mlsanity.com/paper/2601.03024/) (2026)
- [FaithFusion: Harmonizing Reconstruction and Generation via Pixel-wise Information Gain](https://mlsanity.com/paper/2511.21113/) (2025)
- [A Bayesian Approach for Task-Specific Next-Best-View Selection with Uncertain Geometry](https://mlsanity.com/paper/2605.05095/) (2026)
- [GO-PRE: Goal-Oriented Next-Best-View Selection via Predictive Rendering Entropy for Active 3D Reconstruction](https://mlsanity.com/paper/2607.29037/) (2026)
- [Camera Splatting for Continuous View Optimization](https://mlsanity.com/paper/2509.15677/) (2025)
- [FluidGaussian: Propagating Simulation-Based Uncertainty Toward Functionally-Intelligent 3D Reconstruction](https://mlsanity.com/paper/2603.21356/) (2026)
- [VCURF: Virtual Camera-based Uncertainty of Radiance Fields](https://mlsanity.com/paper/2610.04076/) (2026)

## Abstract (arXiv)

Radiance fields are powerful and, hence, popular models for representing the appearance of complex scenes. Yet, constructing them based on image observations gives rise to ambiguities and uncertainties. We propose a versatile approach for learning Gaussian radiance fields with explicit and fine-grained uncertainty estimates that impose only little additional cost compared to uncertainty-agnostic training. Our key observation is that uncertainties can be modeled as a low-dimensional manifold in the space of radiance field parameters that is highly amenable to Monte Carlo sampling. Importantly, our uncertainties are differentiable and, thus, allow for gradient-based optimization of subsequent captures that optimally reduce ambiguities. We demonstrate state-of-the-art performance on next-best-view planning tasks, including high-dimensional illumination planning for optimal radiance field relighting quality.