# AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields

arXiv: https://arxiv.org/abs/2207.10312 · HTML: https://mlsanity.com/paper/2207.10312/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 46 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 30 of 39)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs](https://mlsanity.com/paper/2112.10703/) (2021)
- [Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering](https://mlsanity.com/paper/2106.02634/) (2021)
- [Mixture of Volumetric Primitives for Efficient Neural Rendering](https://mlsanity.com/paper/2103.01954/) (2021)
- [Neural Radiance Flow for 4D View Synthesis and Video Processing](https://mlsanity.com/paper/2012.09790/) (2020)
- [DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks](https://mlsanity.com/paper/2103.03231/) (2021)
- [Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering](https://mlsanity.com/paper/2109.01847/) (2021)

## Built on by (most cited first, 30 of 68)

- [MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes](https://mlsanity.com/paper/2302.12249/) (2023)
- [Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle](https://mlsanity.com/paper/2312.03431/) (2023)
- [HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling](https://mlsanity.com/paper/2301.02238/) (2023)
- [NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields](https://mlsanity.com/paper/2210.15947/) (2022)
- [Watch Your Steps: Local Image and Scene Editing by Text Instructions](https://mlsanity.com/paper/2308.08947/) (2023)
- [SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields](https://mlsanity.com/paper/2212.02501/) (2022)
- [Reference-guided Controllable Inpainting of Neural Radiance Fields](https://mlsanity.com/paper/2304.09677/) (2023)
- [NeRFLiX: High-Quality Neural View Synthesis by Learning a Degradation-Driven Inter-viewpoint MiXer](https://mlsanity.com/paper/2303.06919/) (2023)
- [Learning Neural Duplex Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2304.10537/) (2023)
- [HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces](https://mlsanity.com/paper/2312.03160/) (2023)
- [4K-NeRF: High Fidelity Neural Radiance Fields at Ultra High Resolutions](https://mlsanity.com/paper/2212.04701/) (2022)
- [VRP-UDF: Towards Unbiased Learning of Unsigned Distance Functions from Multi-view Images with Volume Rendering Priors](https://mlsanity.com/paper/2407.16396/) (2024)
- [HollowNeRF: Pruning Hashgrid-Based NeRFs with Trainable Collision Mitigation](https://mlsanity.com/paper/2308.10122/) (2023)
- [Improving Physics-Augmented Continuum Neural Radiance Field-Based Geometry-Agnostic System Identification with Lagrangian Particle Optimization](https://mlsanity.com/paper/2406.04155/) (2024)
- [SteerNeRF: Accelerating NeRF Rendering via Smooth Viewpoint Trajectory](https://mlsanity.com/paper/2212.08476/) (2022)
- [Point-DynRF: Point-based Dynamic Radiance Fields from a Monocular Video](https://mlsanity.com/paper/2310.09647/) (2023)
- [What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs](https://mlsanity.com/paper/2401.02411/) (2024)
- [EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering](https://mlsanity.com/paper/2412.07293/) (2024)
- [Dynamic PlenOctree for Adaptive Sampling Refinement in Explicit NeRF](https://mlsanity.com/paper/2307.15333/) (2023)
- [From NeRFLiX to NeRFLiX++: A General NeRF-Agnostic Restorer Paradigm](https://mlsanity.com/paper/2306.06388/) (2023)
- [MIMO-NeRF: Fast Neural Rendering with Multi-input Multi-output Neural Radiance Fields](https://mlsanity.com/paper/2310.01821/) (2023)
- [FPO++: Efficient Encoding and Rendering of Dynamic Neural Radiance Fields by Analyzing and Enhancing Fourier PlenOctrees](https://mlsanity.com/paper/2310.20710/) (2023)
- [SafeguardGS: 3D Gaussian Primitive Pruning While Avoiding Catastrophic Scene Destruction](https://mlsanity.com/paper/2405.17793/) (2024)
- [SMURF: Continuous Dynamics for Motion-Deblurring Radiance Fields](https://mlsanity.com/paper/2403.07547/) (2024)
- [Multi-task View Synthesis with Neural Radiance Fields](https://mlsanity.com/paper/2309.17450/) (2023)
- [Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations](https://mlsanity.com/paper/2310.17880/) (2023)
- [LAENeRF: Local Appearance Editing for Neural Radiance Fields](https://mlsanity.com/paper/2312.09913/) (2023)
- [Volumetric Surfaces: Representing Fuzzy Geometries with Layered Meshes](https://mlsanity.com/paper/2409.02482/) (2024)
- [MF-NeRF: Memory Efficient NeRF with Mixed-Feature Hash Table](https://mlsanity.com/paper/2304.12587/) (2023)
- [AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization](https://mlsanity.com/paper/2308.10001/) (2023)

## Abstract (arXiv)

Novel view synthesis has recently been revolutionized by learning neural radiance fields directly from sparse observations. However, rendering images with this new paradigm is slow due to the fact that an accurate quadrature of the volume rendering equation requires a large number of samples for each ray. Previous work has mainly focused on speeding up the network evaluations that are associated with each sample point, e.g., via caching of radiance values into explicit spatial data structures, but this comes at the expense of model compactness. In this paper, we propose a novel dual-network architecture that takes an orthogonal direction by learning how to best reduce the number of required sample points. To this end, we split our network into a sampling and shading network that are jointly trained. Our training scheme employs fixed sample positions along each ray, and incrementally introduces sparsity throughout training to achieve high quality even at low sample counts. After fine-tuning with the target number of samples, the resulting compact neural representation can be rendered in real-time. Our experiments demonstrate that our approach outperforms concurrent compact neural representations in terms of quality and frame rate and performs on par with highly efficient hybrid representations. Code and supplementary material is available at https://thomasneff.github.io/adanerf.