mlsanityData to 7 Oct 2026

PlenOctrees for Real-time Rendering of Neural Radiance Fields

arXiv 2103.14024 ↗First posted 25 Mar 2021cs.CV cs.GRcited by 904

Not on the map. Why?

Cited by 22 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce a method to render Neural Radiance Fields (NeRFs) in real time using PlenOctrees, an octree-based 3D representation which supports view-dependent effects. Our method can render 800x800 images at more than 150 FPS, which is over 3000 times faster than conventional NeRFs. We do so without sacrificing quality while preserving the ability of NeRFs to perform free-viewpoint rendering of scenes with arbitrary geometry and view-dependent effects. Real-time performance is achieved by pre-tabulating the NeRF into a PlenOctree. In order to preserve view-dependent effects such as specularities, we factorize the appearance via closed-form spherical basis functions. Specifically, we show that it is possible to train NeRFs to predict a spherical harmonic representation of radiance, removing the viewing direction as an input to the neural network. Furthermore, we show that PlenOctrees can be directly optimized to further minimize the reconstruction loss, which leads to equal or better quality compared to competing methods. Moreover, this octree optimization step can be used to reduce the training time, as we no longer need to wait for the NeRF training to converge fully. Our real-time neural rendering approach may potentially enable new applications such as 6-DOF industrial and product visualizations, as well as next generation AR/VR systems. PlenOctrees are amenable to in-browser rendering as well; please visit the project page for the interactive online demo, as well as video and code: https://alexyu.net/plenoctrees

Key facts. Cited by 904 papers in the map (data through 7 Oct 2026). Builds on 33 of its 64 references that are in the map. First posted on arXiv 25 Mar 2021.

Builds on

33 of 64 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  3. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  4. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  5. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  6. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  7. 2020Neural Sparse Voxel Fieldscited by 959
  8. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  9. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  10. 2020Convolutional Occupancy Networkscited by 760
  11. 2020Implicit Geometric Regularization for Learning Shapescited by 707
  12. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
21 more
  1. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  2. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  3. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  4. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  5. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
  6. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  7. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  8. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  9. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  10. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  11. 2020Learned Initializations for Optimizing Coordinate-Based Neural Representationscited by 240
  12. 2021NeX: Real-time View Synthesis with Neural Basis Expansioncited by 215
  13. 2020AutoInt: Automatic Integration for Fast Neural Volume Renderingcited by 204
  14. 2019Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Imagecited by 202
  15. 2017Learning a Multi-View Stereo Machinecited by 194
  16. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  17. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  18. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  19. 2020DeRF: Decomposed Radiance Fieldscited by 163
  20. 2017Hierarchical Surface Prediction for 3D Object Reconstructioncited by 89
  21. 2020Monocular Real-Time Volumetric Performance Capturecited by 74

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 56; 2022: 177; 2023: 275; 2024: 233; 2025: 115; 2026: 48.

904 papers.

Sort:

By area:

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  3. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  4. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  5. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  6. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  7. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  8. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,061
  9. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  10. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 753
  11. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  12. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
38 more
  1. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  2. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  3. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  4. 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 661
  5. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 611
  6. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  7. 2021Neural Fields in Visual Computing and Beyondcited by 539
  8. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  9. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  10. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 465
  11. 2023LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPScited by 420
  12. 2021Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughscited by 396
  13. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  14. 2021Advances in Neural Renderingcited by 365
  15. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  16. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  17. 2022Fast Dynamic Radiance Fields with Time-Aware Neural Voxelscited by 344
  18. 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 341
  19. 2023SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenescited by 339
  20. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  21. 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 308
  22. 2023Text-to-3D using Gaussian Splattingcited by 299
  23. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  24. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  25. 2024DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalizationcited by 256
  26. 2023SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesiscited by 247
  27. 2023Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesiscited by 232
  28. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  29. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  30. 2024Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussianscited by 225
  31. 2021Editing Conditional Radiance Fieldscited by 222
  32. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  33. 2023BakedSDF: Meshing Neural SDFs for Real-Time View Synthesiscited by 220
  34. 2022Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Renderingcited by 217
  35. 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 216
  36. 2023MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenescited by 215
  37. 20243D Gaussian Splatting as Markov Chain Monte Carlocited by 210
  38. 2024VastGaussian: Vast 3D Gaussians for Large Scene Reconstructioncited by 209

50 most cited of 904 All 904 (JSON)