mlsanityData to 14 Oct 2026

Light Field Neural Rendering

arXiv 2112.09687 ↗First posted 17 Dec 2021cs.CVcited by 108

Not on the map. Why?

Abstract (arXiv)

Classical light field rendering for novel view synthesis can accurately reproduce view-dependent effects such as reflection, refraction, and translucency, but requires a dense view sampling of the scene. Methods based on geometric reconstruction need only sparse views, but cannot accurately model non-Lambertian effects. We introduce a model that combines the strengths and mitigates the limitations of these two directions. By operating on a four-dimensional representation of the light field, our model learns to represent view-dependent effects accurately. By enforcing geometric constraints during training and inference, the scene geometry is implicitly learned from a sparse set of views. Concretely, we introduce a two-stage transformer-based model that first aggregates features along epipolar lines, then aggregates features along reference views to produce the color of a target ray. Our model outperforms the state-of-the-art on multiple forward-facing and 360deg datasets, with larger margins on scenes with severe view-dependent variations.

Key facts. Cited by 108 papers in the map (data through 14 Oct 2026). Builds on 47 of its 86 references that are in the map. First posted on arXiv 17 Dec 2021.

Builds on

47 of 86 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2017Graph Attention Networkscited by 8,923
  6. 2016Layer Normalizationcited by 5,669
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  8. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  9. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  10. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  12. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
35 more
  1. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  2. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  3. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
  4. 2020Convolutional Occupancy Networkscited by 761
  5. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  6. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  7. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  8. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  9. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
  10. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  11. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  12. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 405
  13. 2020Local Implicit Grid Representations for 3D Scenescited by 373
  14. 2021Advances in Neural Renderingcited by 365
  15. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 333
  16. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 332
  17. 2019Neural Point-Based Graphicscited by 327
  18. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  19. 2020Neural Radiance Flow for 4D View Synthesis and Video Processingcited by 270
  20. 2020Free View Synthesiscited by 231
  21. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 227
  22. 2021NeX: Real-time View Synthesis with Neural Basis Expansioncited by 216
  23. 2020AutoInt: Automatic Integration for Fast Neural Volume Renderingcited by 204
  24. 2021GeoNeRF: Generalizing NeRF with Geometry Priorscited by 196
  25. 2021Neural Rays for Occlusion-aware Image-based Renderingcited by 192
  26. 2020Stable View Synthesiscited by 165
  27. 2016Learning-Based View Synthesis for Light Field Camerascited by 145
  28. 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 100
  29. 2018Extreme View Synthesiscited by 94
  30. 2019Controlling Neural Level Setscited by 76
  31. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  32. 2017Learning to Synthesize a 4D RGBD Light Field from a Single Imagecited by 60
  33. 2021Revisiting Light Field Rendering with Deep Anti-Aliasing Neural Networkcited by 13
  34. 2018IGNOR: Image-guided Neural Object Renderingcited by 12
  35. 2021Transformers: "The End of History" for NLP?cited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 11; 2023: 54; 2024: 21; 2025: 14; 2026: 8.

108 papers.

Sort:

By area:

  1. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
  2. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
  3. 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 344
  4. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 279
  5. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
  6. 20233DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splattingcited by 195
  7. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 182
  8. 2023HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Samplingcited by 174
  9. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
  10. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 139
  11. 2024latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstructioncited by 118
  12. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 116
38 more
  1. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 116
  2. 2022Is Attention All That NeRF Needs?cited by 113
  3. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  4. 2025Cameras as Relative Positional Encodingcited by 105
  5. 2023Learning to Render Novel Views from Wide-Baseline Stereo Pairscited by 104
  6. 20234K4D: Real-Time 4D View Synthesis at 4K Resolutioncited by 91
  7. 2023Structure-Aware Sparse-View X-ray 3D Reconstructioncited by 72
  8. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72
  9. 2023DBARF: Deep Bundle-Adjusting Generalizable Neural Radiance Fieldscited by 60
  10. 2024DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensuscited by 51
  11. 2023Explicit Correspondence Matching for Generalizable Neural Radiance Fieldscited by 46
  12. 20243DRealCar: An In-the-wild RGB-D Car Dataset with 360-degree Viewscited by 39
  13. 2023DELIFFAS: Deformable Light Fields for Fast Avatar Synthesiscited by 36
  14. 2024Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstructioncited by 32
  15. 2024G3R: Gradient Guided Generalizable Reconstructioncited by 31
  16. 2023NeRFLiX: High-Quality Neural View Synthesis by Learning a Degradation-Driven Inter-viewpoint MiXercited by 29
  17. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  18. 2023Pseudo-Generalized Dynamic View Synthesis from a Videocited by 27
  19. 2023Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Viewscited by 25
  20. 2022360FusionNeRF: Panoramic Neural Radiance Fields with Joint Guidancecited by 23
  21. 2023Pointersect: Neural Rendering with Cloud-Ray Intersectioncited by 23
  22. 2023Fast View Synthesis of Casual Videos with Soup-of-Planescited by 22
  23. 2023Learning Non-Local Spatial-Angular Correlation for Light Field Image Super-Resolutioncited by 21
  24. 2022Attention Beats Concatenation for Conditioning Neural Fieldscited by 20
  25. 2023GP-NeRF: Generalized Perception NeRF for Context-Aware 3D Scene Understandingcited by 20
  26. 2024GGRt: Towards Pose-free Generalizable 3D Gaussian Splatting in Real-timecited by 20
  27. 2023Local Implicit Ray Function for Generalizable Radiance Field Representationcited by 19
  28. 2023Multiscale Tensor Decomposition and Rendering Equation Encoding for View Synthesiscited by 18
  29. 2024Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesiscited by 18
  30. 20257DGS: Unified Spatial-Temporal-Angular Gaussian Splattingcited by 18
  31. 2023Re-ReND: Real-time Rendering of NeRFs across Devicescited by 17
  32. 2023Perceptual Quality Assessment of NeRF and Neural View Synthesis Methods for Front-Facing Viewscited by 17
  33. 2023Accelerating Neural Field Training via Soft Miningcited by 15
  34. 2024Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3Dcited by 15
  35. 2023FlowIBR: Leveraging Pre-Training for Efficient Neural Image-Based Rendering of Dynamic Scenescited by 14
  36. 20254D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Timecited by 14
  37. 2023RayDF: Neural Ray-surface Distance Fields with Multi-view Consistencycited by 12
  38. 2023Neural Spectro-polarimetric Fieldscited by 10

50 most cited of 108 All 108 (JSON)