# Relit-NeuLF: Efficient Relighting and Novel View Synthesis via Neural 4D Light Field

arXiv: https://arxiv.org/abs/2310.14642 · HTML: https://mlsanity.com/paper/2310.14642/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 71 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 30 of 30)

- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering](https://mlsanity.com/paper/2106.02634/) (2021)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Stereo Magnification: Learning View Synthesis using Multiplane Images](https://mlsanity.com/paper/1805.09817/) (2018)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [Modeling Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2204.06837/) (2022)
- [Single Image Portrait Relighting](https://mlsanity.com/paper/1905.00824/) (2019)
- [NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild](https://mlsanity.com/paper/2110.07604/) (2021)
- [IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images](https://mlsanity.com/paper/2204.02232/) (2022)
- [SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild](https://mlsanity.com/paper/1712.01261/) (2017)
- [SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections](https://mlsanity.com/paper/2205.15768/) (2022)
- [Fast Spatially-Varying Indoor Lighting Estimation](https://mlsanity.com/paper/1906.03799/) (2019)
- [Neural Light Transport for Relighting and View Synthesis](https://mlsanity.com/paper/2008.03806/) (2020)
- [PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereo](https://mlsanity.com/paper/2207.11406/) (2022)
- [Two-shot Spatially-varying BRDF and Shape Estimation](https://mlsanity.com/paper/2004.00403/) (2020)
- [Learning Neural Light Fields with Ray-Space Embedding Networks](https://mlsanity.com/paper/2112.01523/) (2021)
- [NeLF: Neural Light-transport Field for Portrait View Synthesis and Relighting](https://mlsanity.com/paper/2107.12351/) (2021)
- [NeuLF: Efficient Novel View Synthesis with Neural 4D Light Field](https://mlsanity.com/paper/2105.07112/) (2021)
- [Vid2Player: Controllable Video Sprites that Behave and Appear like Professional Tennis Players](https://mlsanity.com/paper/2008.04524/) (2020)
- [Relighting Humans: Occlusion-Aware Inverse Rendering for Full-Body Human Images](https://mlsanity.com/paper/1908.02714/) (2019)
- [Robust 3D Human Motion Reconstruction Via Dynamic Template Construction](https://mlsanity.com/paper/1801.10434/) (2018)
- [4D Human Body Correspondences from Panoramic Depth Maps](https://mlsanity.com/paper/1810.05340/) (2018)
- [Harnessing Low-Frequency Neural Fields for Few-Shot View Synthesis](https://mlsanity.com/paper/2303.08370/) (2023)

## Built on by (most cited first, 7 of 7)

- [Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis](https://mlsanity.com/paper/2312.16812/) (2023)
- [DifFRelight: Diffusion-Based Facial Performance Relighting](https://mlsanity.com/paper/2410.08188/) (2024)
- [NeAR: Coupled Neural Asset-Renderer Stack](https://mlsanity.com/paper/2511.18600/) (2025)
- [MERLiN: Single-Shot Material Estimation and Relighting for Photometric Stereo](https://mlsanity.com/paper/2409.00674/) (2024)
- [DualMat: PBR Material Estimation via Coherent Dual-Path Diffusion](https://mlsanity.com/paper/2508.05060/) (2025)
- [Time-Archival Camera Virtualization for Sports and Visual Performances](https://mlsanity.com/paper/2602.15181/) (2026)
- [HAFMat: Hybrid Priors Guided Adaptive Fusion for Single-Image Human Material Estimation](https://mlsanity.com/paper/2606.16323/) (2026)

## Abstract (arXiv)

In this paper, we address the problem of simultaneous relighting and novel view synthesis of a complex scene from multi-view images with a limited number of light sources. We propose an analysis-synthesis approach called Relit-NeuLF. Following the recent neural 4D light field network (NeuLF), Relit-NeuLF first leverages a two-plane light field representation to parameterize each ray in a 4D coordinate system, enabling efficient learning and inference. Then, we recover the spatially-varying bidirectional reflectance distribution function (SVBRDF) of a 3D scene in a self-supervised manner. A DecomposeNet learns to map each ray to its SVBRDF components: albedo, normal, and roughness. Based on the decomposed BRDF components and conditioning light directions, a RenderNet learns to synthesize the color of the ray. To self-supervise the SVBRDF decomposition, we encourage the predicted ray color to be close to the physically-based rendering result using the microfacet model. Comprehensive experiments demonstrate that the proposed method is efficient and effective on both synthetic data and real-world human face data, and outperforms the state-of-the-art results. We publicly released our code on GitHub. You can find it here: https://github.com/oppo-us-research/RelitNeuLF