mlsanityData to 7 Oct 2026

Neural Lumigraph Rendering

arXiv 2103.11571 ↗First posted 22 Mar 2021cs.CV cs.GRcited by 105

Not on the map. Why?

Abstract (arXiv)

Novel view synthesis is a challenging and ill-posed inverse rendering problem. Neural rendering techniques have recently achieved photorealistic image quality for this task. State-of-the-art (SOTA) neural volume rendering approaches, however, are slow to train and require minutes of inference (i.e., rendering) time for high image resolutions. We adopt high-capacity neural scene representations with periodic activations for jointly optimizing an implicit surface and a radiance field of a scene supervised exclusively with posed 2D images. Our neural rendering pipeline accelerates SOTA neural volume rendering by about two orders of magnitude and our implicit surface representation is unique in allowing us to export a mesh with view-dependent texture information. Thus, like other implicit surface representations, ours is compatible with traditional graphics pipelines, enabling real-time rendering rates, while achieving unprecedented image quality compared to other surface methods. We assess the quality of our approach using existing datasets as well as high-quality 3D face data captured with a custom multi-camera rig.

Key facts. Cited by 105 papers in the map (data through 7 Oct 2026). Builds on 32 of its 63 references that are in the map. First posted on arXiv 22 Mar 2021.

Builds on

32 of 63 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  7. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  8. 2020Neural Sparse Voxel Fieldscited by 959
  9. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  10. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  11. 2020Convolutional Occupancy Networkscited by 760
  12. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
20 more
  1. 2020Implicit Geometric Regularization for Learning Shapescited by 707
  2. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  3. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  4. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  5. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
  6. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  7. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  8. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  9. 2020State of the Art on Neural Renderingcited by 276
  10. 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
  11. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  12. 2020Free View Synthesiscited by 231
  13. 2020MetaSDF: Meta-learning Signed Distance Functionscited by 214
  14. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  15. 2020AutoInt: Automatic Integration for Fast Neural Volume Renderingcited by 204
  16. 2019DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracingcited by 179
  17. 2019SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimizationcited by 159
  18. 2020Neural Light Transport for Relighting and View Synthesiscited by 73
  19. 2020MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Imagescited by 64
  20. 2020On the Effectiveness of Weight-Encoded Neural Implicit 3D Shapescited by 29

Built on by

Cited by year

Citing papers per year: 2021: 22; 2022: 38; 2023: 33; 2024: 10; 2025: 2.

105 papers.

Sort:

By area:

  1. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  2. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  3. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  4. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  5. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  6. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  7. 2021Neural Fields in Visual Computing and Beyondcited by 539
  8. 2021Advances in Neural Renderingcited by 365
  9. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  10. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  11. 2022NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstructioncited by 311
  12. 20223D Neural Field Generation using Triplane Diffusioncited by 274
38 more
  1. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 230
  2. 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 216
  3. 2023MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenescited by 215
  4. 2021Neural Head Avatars from Monocular RGB Videoscited by 212
  5. 2021Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decompositioncited by 204
  6. 2021Neural Rays for Occlusion-aware Image-based Renderingcited by 192
  7. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 183
  8. 2022SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Viewscited by 175
  9. 2022Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentationcited by 152
  10. 2021BACON: Band-limited Coordinate Networks for Multiscale Scene Representationcited by 142
  11. 2023GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesiscited by 131
  12. 2022Generalizable Patch-Based Neural Renderingcited by 121
  13. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
  14. 2022Generative Neural Articulated Radiance Fieldscited by 101
  15. 2022Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstructioncited by 92
  16. 20234K4D: Real-Time 4D View Synthesis at 4K Resolutioncited by 91
  17. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
  18. 2022DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Renderingcited by 80
  19. 20243DGSR: Implicit Surface Reconstruction with 3D Gaussian Splattingcited by 78
  20. 2024Fully Explicit Dynamic Gaussian Splattingcited by 75
  21. 2022Human Performance Modeling and Rendering via Neural Animated Meshcited by 64
  22. 2022NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologiescited by 62
  23. 2021DIVeR: Real-time and Accurate Neural Radiance Fields with Deterministic Integration for Volume Renderingcited by 60
  24. 2022Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Imagescited by 59
  25. 2023Im4D: High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenescited by 55
  26. 2021Generative Occupancy Fields for 3D Surface-Aware Image Synthesiscited by 51
  27. 2022Critical Regularizations for Neural Surface Reconstruction in the Wildcited by 51
  28. 2022NeuPhysics: Editable Neural Geometry and Physics from Monocular Videoscited by 50
  29. 2021Learning Neural Light Fields with Ray-Space Embedding Networkscited by 49
  30. 2022Multi-View Mesh Reconstruction with Neural Deferred Shadingcited by 44
  31. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  32. 2022A Level Set Theory for Neural Implicit Evolution under Explicit Flowscited by 42
  33. 2022Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesiscited by 40
  34. 2023MixNeRF: Modeling a Ray with Mixture Density for Novel View Synthesis from Sparse Inputscited by 40
  35. 2023NeUDF: Leaning Neural Unsigned Distance Fields with Volume Renderingcited by 40
  36. 2022FDNeRF: Few-shot Dynamic Neural Radiance Fields for Face Reconstruction and Expression Editingcited by 38
  37. 2023HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Fieldcited by 37
  38. 2023Real-Time Neural Rasterization for Large Scenescited by 37

50 most cited of 105 All 105 (JSON)