# NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimation

arXiv: https://arxiv.org/abs/2303.17147 · HTML: https://mlsanity.com/paper/2303.17147/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 30 Mar 2023.

## Builds on (most cited first, 23 of 23)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [NeILF: Neural Incident Light Field for Physically-based Material Estimation](https://mlsanity.com/paper/2203.07182/) (2022)
- [Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images](https://mlsanity.com/paper/2007.09892/) (2020)
- [Visibility-aware Multi-view Stereo Network](https://mlsanity.com/paper/2008.07928/) (2020)
- [Learning Signed Distance Field for Multi-view Surface Reconstruction](https://mlsanity.com/paper/2108.09964/) (2021)
- [Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images](https://mlsanity.com/paper/2003.12642/) (2020)
- [Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination](https://mlsanity.com/paper/2003.08367/) (2020)
- [Inverse Path Tracing for Joint Material and Lighting Estimation](https://mlsanity.com/paper/1903.07145/) (2019)
- [Critical Regularizations for Neural Surface Reconstruction in the Wild](https://mlsanity.com/paper/2206.03087/) (2022)
- [Neural Reflectance for Shape Recovery with Shadow Handling](https://mlsanity.com/paper/2203.12909/) (2022)
- [Seeing the World in a Bag of Chips](https://mlsanity.com/paper/2001.04642/) (2020)
- [ShadowNeuS: Neural SDF Reconstruction by Shadow Ray Supervision](https://mlsanity.com/paper/2211.14086/) (2022)

## Built on by (most cited first, 30 of 74)

- [Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing](https://mlsanity.com/paper/2311.16043/) (2023)
- [Neural Gaffer: Relighting Any Object via Diffusion](https://mlsanity.com/paper/2406.07520/) (2024)
- [GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization](https://mlsanity.com/paper/2312.05133/) (2023)
- [IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination](https://mlsanity.com/paper/2404.11593/) (2024)
- [DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models](https://mlsanity.com/paper/2405.17176/) (2024)
- [GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians](https://mlsanity.com/paper/2402.10483/) (2024)
- [A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis](https://mlsanity.com/paper/2409.08947/) (2024)
- [DifFRelight: Diffusion-Based Facial Performance Relighting](https://mlsanity.com/paper/2410.08188/) (2024)
- [SpecNeRF: Gaussian Directional Encoding for Specular Reflections](https://mlsanity.com/paper/2312.13102/) (2023)
- [Ref-GS: Directional Factorization for 2D Gaussian Splatting](https://mlsanity.com/paper/2412.00905/) (2024)
- [NeISF: Neural Incident Stokes Field for Geometry and Material Estimation](https://mlsanity.com/paper/2311.13187/) (2023)
- [SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Rendering](https://mlsanity.com/paper/2504.06815/) (2025)
- [IRIS: Inverse Rendering of Indoor Scenes from Low Dynamic Range Images](https://mlsanity.com/paper/2401.12977/) (2024)
- [Neural LightRig: Unlocking Accurate Object Normal and Material Estimation with Multi-Light Diffusion](https://mlsanity.com/paper/2412.09593/) (2024)
- [SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild](https://mlsanity.com/paper/2401.10171/) (2024)
- [PRTGS: Precomputed Radiance Transfer of Gaussian Splats for Real-Time High-Quality Relighting](https://mlsanity.com/paper/2408.03538/) (2024)
- [iVR-GS: Inverse Volume Rendering for Explorable Visualization via Editable 3D Gaussian Splatting](https://mlsanity.com/paper/2504.17954/) (2025)
- [RNG: Relightable Neural Gaussians](https://mlsanity.com/paper/2409.19702/) (2024)
- [TransparentGS: Fast Inverse Rendering of Transparent Objects with Gaussians](https://mlsanity.com/paper/2504.18768/) (2025)
- [Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects](https://mlsanity.com/paper/2305.17929/) (2023)
- [GS-ID: Illumination Decomposition on Gaussian Splatting via Adaptive Light Aggregation and Diffusion-Guided Material Priors](https://mlsanity.com/paper/2408.08524/) (2024)
- [GlossyGS: Inverse Rendering of Glossy Objects with 3D Gaussian Splatting](https://mlsanity.com/paper/2410.13349/) (2024)
- [PBR-NeRF: Inverse Rendering with Physics-Based Neural Fields](https://mlsanity.com/paper/2412.09680/) (2024)
- [Gaussian Splatting with Discretized SDF for Relightable Assets](https://mlsanity.com/paper/2507.15629/) (2025)
- [Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering](https://mlsanity.com/paper/2409.05867/) (2024)
- [GS-ROR2: Bidirectional-guided 3DGS and SDF for Reflective Object Relighting and Reconstruction](https://mlsanity.com/paper/2406.18544/) (2024)
- [RTR-GS: 3D Gaussian Splatting for Inverse Rendering with Radiance Transfer and Reflection](https://mlsanity.com/paper/2507.07733/) (2025)
- [NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics](https://mlsanity.com/paper/2411.10189/) (2024)
- [3D Gaussian Inverse Rendering with Approximated Global Illumination](https://mlsanity.com/paper/2504.01358/) (2025)
- [Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering](https://mlsanity.com/paper/2410.20593/) (2024)

## Abstract (arXiv)

We present a novel differentiable rendering framework for joint geometry, material, and lighting estimation from multi-view images. In contrast to previous methods which assume a simplified environment map or co-located flashlights, in this work, we formulate the lighting of a static scene as one neural incident light field (NeILF) and one outgoing neural radiance field (NeRF). The key insight of the proposed method is the union of the incident and outgoing light fields through physically-based rendering and inter-reflections between surfaces, making it possible to disentangle the scene geometry, material, and lighting from image observations in a physically-based manner. The proposed incident light and inter-reflection framework can be easily applied to other NeRF systems. We show that our method can not only decompose the outgoing radiance into incident lights and surface materials, but also serve as a surface refinement module that further improves the reconstruction detail of the neural surface. We demonstrate on several datasets that the proposed method is able to achieve state-of-the-art results in terms of geometry reconstruction quality, material estimation accuracy, and the fidelity of novel view rendering.