# ProbNeRF: Uncertainty-Aware Inference of 3D Shapes from 2D Images

arXiv: https://arxiv.org/abs/2210.17415 · HTML: https://mlsanity.com/paper/2210.17415/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 62 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 30 of 40)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures](https://mlsanity.com/paper/2208.00277/) (2022)
- [Novel View Synthesis with Diffusion Models](https://mlsanity.com/paper/2210.04628/) (2022)
- [Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations](https://mlsanity.com/paper/2111.13152/) (2021)
- [From data to functa: Your data point is a function and you can treat it like one](https://mlsanity.com/paper/2201.12204/) (2022)
- [GRF: Learning a General Radiance Field for 3D Representation and Rendering](https://mlsanity.com/paper/2010.04595/) (2020)
- [CodeNeRF: Disentangled Neural Radiance Fields for Object Categories](https://mlsanity.com/paper/2109.01750/) (2021)
- [A Complete Recipe for Stochastic Gradient MCMC](https://mlsanity.com/paper/1506.04696/) (2015)
- [GAUDI: A Neural Architect for Immersive 3D Scene Generation](https://mlsanity.com/paper/2207.13751/) (2022)
- [Composing graphical models with neural networks for structured representations and fast inference](https://mlsanity.com/paper/1603.06277/) (2016)
- [NeRF-VAE: A Geometry Aware 3D Scene Generative Model](https://mlsanity.com/paper/2104.00587/) (2021)
- [Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference](https://mlsanity.com/paper/1703.09194/) (2017)
- [ShaRF: Shape-conditioned Radiance Fields from a Single View](https://mlsanity.com/paper/2102.08860/) (2021)
- [LOLNeRF: Learn from One Look](https://mlsanity.com/paper/2111.09996/) (2021)

## Built on by (most cited first, 12 of 12)

- [Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields](https://mlsanity.com/paper/2309.03185/) (2023)
- [Neural Visibility Field for Uncertainty-Driven Active Mapping](https://mlsanity.com/paper/2406.06948/) (2024)
- [Scalable Spatiotemporal Prediction with Bayesian Neural Fields](https://mlsanity.com/paper/2403.07657/) (2024)
- [Learning a Diffusion Prior for NeRFs](https://mlsanity.com/paper/2304.14473/) (2023)
- [PhysGS: Bayesian-Inferred Gaussian Splatting for Physical Property Estimation](https://mlsanity.com/paper/2511.18570/) (2025)
- [3D Neural Embedding Likelihood: Probabilistic Inverse Graphics for Robust 6D Pose Estimation](https://mlsanity.com/paper/2302.03744/) (2023)
- [Peering into the Unknown: Active View Selection with Neural Uncertainty Maps for 3D Reconstruction](https://mlsanity.com/paper/2506.14856/) (2025)
- [Uncertainty-aware Active Learning of NeRF-based Object Models for Robot Manipulators using Visual and Re-orientation Actions](https://mlsanity.com/paper/2404.01812/) (2024)
- [BayesSDF: Surface-Based Laplacian Uncertainty Estimation for 3D Geometry with Neural Signed Distance Fields](https://mlsanity.com/paper/2507.06269/) (2025)
- [FaithFusion: Harmonizing Reconstruction and Generation via Pixel-wise Information Gain](https://mlsanity.com/paper/2511.21113/) (2025)
- [Printing the Underdetermined: Materializing Multi-solutionness in Figurative Paintings](https://mlsanity.com/paper/2609.19782/) (2026)
- [Matisse: Evidence-Space Reasoning for Active 3D Reconstruction](https://mlsanity.com/paper/2609.38746/) (2026)

## Abstract (arXiv)

The problem of inferring object shape from a single 2D image is underconstrained. Prior knowledge about what objects are plausible can help, but even given such prior knowledge there may still be uncertainty about the shapes of occluded parts of objects. Recently, conditional neural radiance field (NeRF) models have been developed that can learn to infer good point estimates of 3D models from single 2D images. The problem of inferring uncertainty estimates for these models has received less attention. In this work, we propose probabilistic NeRF (ProbNeRF), a model and inference strategy for learning probabilistic generative models of 3D objects' shapes and appearances, and for doing posterior inference to recover those properties from 2D images. ProbNeRF is trained as a variational autoencoder, but at test time we use Hamiltonian Monte Carlo (HMC) for inference. Given one or a few 2D images of an object (which may be partially occluded), ProbNeRF is able not only to accurately model the parts it sees, but also to propose realistic and diverse hypotheses about the parts it does not see. We show that key to the success of ProbNeRF are (i) a deterministic rendering scheme, (ii) an annealed-HMC strategy, (iii) a hypernetwork-based decoder architecture, and (iv) doing inference over a full set of NeRF weights, rather than just a low-dimensional code.