# Learning Neural Light Fields with Ray-Space Embedding Networks

arXiv: https://arxiv.org/abs/2112.01523 · HTML: https://mlsanity.com/paper/2112.01523/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 71 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 30 of 44)

- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [In-Place Scene Labelling and Understanding with Implicit Scene Representation](https://mlsanity.com/paper/2103.15875/) (2021)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering](https://mlsanity.com/paper/2106.02634/) (2021)
- [Neural Point-Based Graphics](https://mlsanity.com/paper/1906.08240/) (2019)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks](https://mlsanity.com/paper/2103.03231/) (2021)
- [Free View Synthesis](https://mlsanity.com/paper/2008.05511/) (2020)

## Built on by (most cited first, 30 of 49)

- [Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis](https://mlsanity.com/paper/2312.16812/) (2023)
- [Generalizable Patch-Based Neural Rendering](https://mlsanity.com/paper/2207.10662/) (2022)
- [4K4D: Real-Time 4D View Synthesis at 4K Resolution](https://mlsanity.com/paper/2310.11448/) (2023)
- [R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis](https://mlsanity.com/paper/2203.17261/) (2022)
- [AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2207.10312/) (2022)
- [Urban Radiance Field Representation with Deformable Neural Mesh Primitives](https://mlsanity.com/paper/2307.10776/) (2023)
- [Im4D: High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenes](https://mlsanity.com/paper/2310.08585/) (2023)
- [GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh](https://mlsanity.com/paper/2404.07991/) (2024)
- [Real-Time Neural Light Field on Mobile Devices](https://mlsanity.com/paper/2212.08057/) (2022)
- [Textured Gaussians for Enhanced 3D Scene Appearance Modeling](https://mlsanity.com/paper/2411.18625/) (2024)
- [ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field](https://mlsanity.com/paper/2211.13226/) (2022)
- [DELIFFAS: Deformable Light Fields for Fast Avatar Synthesis](https://mlsanity.com/paper/2310.11449/) (2023)
- [Eikonal Fields for Refractive Novel-View Synthesis](https://mlsanity.com/paper/2202.00948/) (2022)
- [Learning Neural Duplex Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2304.10537/) (2023)
- [Pseudo-Generalized Dynamic View Synthesis from a Video](https://mlsanity.com/paper/2310.08587/) (2023)
- [Learning Non-Local Spatial-Angular Correlation for Light Field Image Super-Resolution](https://mlsanity.com/paper/2302.08058/) (2023)
- [Dynamic Mesh-Aware Radiance Fields](https://mlsanity.com/paper/2309.04581/) (2023)
- [SteerNeRF: Accelerating NeRF Rendering via Smooth Viewpoint Trajectory](https://mlsanity.com/paper/2212.08476/) (2022)
- [OmnimatteRF: Robust Omnimatte with 3D Background Modeling](https://mlsanity.com/paper/2309.07749/) (2023)
- [Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variation](https://mlsanity.com/paper/2412.15211/) (2024)
- [Multiscale Representation for Real-Time Anti-Aliasing Neural Rendering](https://mlsanity.com/paper/2304.10075/) (2023)
- [Global Latent Neural Rendering](https://mlsanity.com/paper/2312.08338/) (2023)
- [LightSpeed: Light and Fast Neural Light Fields on Mobile Devices](https://mlsanity.com/paper/2310.16832/) (2023)
- [3D Motion Magnification: Visualizing Subtle Motions with Time Varying Radiance Fields](https://mlsanity.com/paper/2308.03757/) (2023)
- [Relit-NeuLF: Efficient Relighting and Novel View Synthesis via Neural 4D Light Field](https://mlsanity.com/paper/2310.14642/) (2023)
- [HR-INR: Continuous Space-Time Video Super-Resolution via Event Camera](https://mlsanity.com/paper/2405.13389/) (2024)
- [Neural Light Spheres for Implicit Image Stitching and View Synthesis](https://mlsanity.com/paper/2409.17924/) (2024)
- [Seeing the World through Your Eyes](https://mlsanity.com/paper/2306.09348/) (2023)
- [Sparse-DeRF: Deblurred Neural Radiance Fields from Sparse View](https://mlsanity.com/paper/2407.06613/) (2024)
- [Scaling Transformer-Based Novel View Synthesis Models with Token Disentanglement and Synthetic Data](https://mlsanity.com/paper/2509.06950/) (2025)

## Abstract (arXiv)

Neural radiance fields (NeRFs) produce state-of-the-art view synthesis results. However, they are slow to render, requiring hundreds of network evaluations per pixel to approximate a volume rendering integral. Baking NeRFs into explicit data structures enables efficient rendering, but results in a large increase in memory footprint and, in many cases, a quality reduction. In this paper, we propose a novel neural light field representation that, in contrast, is compact and directly predicts integrated radiance along rays. Our method supports rendering with a single network evaluation per pixel for small baseline light field datasets and can also be applied to larger baselines with only a few evaluations per pixel. At the core of our approach is a ray-space embedding network that maps the 4D ray-space manifold into an intermediate, interpolable latent space. Our method achieves state-of-the-art quality on dense forward-facing datasets such as the Stanford Light Field dataset. In addition, for forward-facing scenes with sparser inputs we achieve results that are competitive with NeRF-based approaches in terms of quality while providing a better speed/quality/memory trade-off with far fewer network evaluations.