# NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illumination

arXiv: https://arxiv.org/abs/2303.16617 · HTML: https://mlsanity.com/paper/2303.16617/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Neural Reflectance Fields for Appearance Acquisition](https://mlsanity.com/paper/2008.03824/) (2020)
- [Modeling Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2204.06837/) (2022)
- [Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising](https://mlsanity.com/paper/2206.03380/) (2022)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [InverseRenderNet: Learning single image inverse rendering](https://mlsanity.com/paper/1811.12328/) (2018)
- [Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images](https://mlsanity.com/paper/2007.09892/) (2020)
- [Neural Light Transport for Relighting and View Synthesis](https://mlsanity.com/paper/2008.03806/) (2020)
- [Inverse Path Tracing for Joint Material and Lighting Estimation](https://mlsanity.com/paper/1903.07145/) (2019)
- [Shape and Material Capture at Home](https://mlsanity.com/paper/2104.06397/) (2021)
- [NeROIC: Neural Rendering of Objects from Online Image Collections](https://mlsanity.com/paper/2201.02533/) (2022)
- [Object-based Illumination Estimation with Rendering-aware Neural Networks](https://mlsanity.com/paper/2008.02514/) (2020)

## Built on by (most cited first, 30 of 38)

- [Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing](https://mlsanity.com/paper/2311.16043/) (2023)
- [GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization](https://mlsanity.com/paper/2312.05133/) (2023)
- [IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination](https://mlsanity.com/paper/2404.11593/) (2024)
- [Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark](https://mlsanity.com/paper/2310.16044/) (2023)
- [Neural-PBIR Reconstruction of Shape, Material, and Illumination](https://mlsanity.com/paper/2304.13445/) (2023)
- [Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Dataset](https://mlsanity.com/paper/2504.08541/) (2025)
- [NeISF: Neural Incident Stokes Field for Geometry and Material Estimation](https://mlsanity.com/paper/2311.13187/) (2023)
- [IRIS: Inverse Rendering of Indoor Scenes from Low Dynamic Range Images](https://mlsanity.com/paper/2401.12977/) (2024)
- [Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects](https://mlsanity.com/paper/2305.17929/) (2023)
- [PBR-NeRF: Inverse Rendering with Physics-Based Neural Fields](https://mlsanity.com/paper/2412.09680/) (2024)
- [Mobile-GS: Real-time Gaussian Splatting for Mobile Devices](https://mlsanity.com/paper/2603.11531/) (2026)
- [GaSLight: Gaussian Splats for Spatially-Varying Lighting in HDR](https://mlsanity.com/paper/2504.10809/) (2025)
- [NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics](https://mlsanity.com/paper/2411.10189/) (2024)
- [NeRF as a Non-Distant Environment Emitter in Physics-based Inverse Rendering](https://mlsanity.com/paper/2402.04829/) (2024)
- [TraM-NeRF: Tracing Mirror and Near-Perfect Specular Reflections through Neural Radiance Fields](https://mlsanity.com/paper/2310.10650/) (2023)
- [Diffusion Posterior Illumination for Ambiguity-aware Inverse Rendering](https://mlsanity.com/paper/2310.00362/) (2023)
- [Differentiable Inverse Rendering with Interpretable Basis BRDFs](https://mlsanity.com/paper/2411.17994/) (2024)
- [SpecGaussian with Latent Features: A High-quality Modeling of the View-dependent Appearance for 3D Gaussian Splatting](https://mlsanity.com/paper/2409.05868/) (2024)
- [MERLiN: Single-Shot Material Estimation and Relighting for Photometric Stereo](https://mlsanity.com/paper/2409.00674/) (2024)
- [Differentiable Light Transport with Gaussian Surfels via Adapted Radiosity for Efficient Relighting and Geometry Reconstruction](https://mlsanity.com/paper/2509.18497/) (2025)
- [RNA: Relightable Neural Assets](https://mlsanity.com/paper/2312.09398/) (2023)
- [ESR-NeRF: Emissive Source Reconstruction Using LDR Multi-view Images](https://mlsanity.com/paper/2404.15707/) (2024)
- [PIR: Photometric Inverse Rendering with Shading Cues Modeling and Surface Reflectance Regularization](https://mlsanity.com/paper/2408.06828/) (2024)
- [NeRFs are Mirror Detectors: Using Structural Similarity for Multi-View Mirror Scene Reconstruction with 3D Surface Primitives](https://mlsanity.com/paper/2501.04074/) (2025)
- [ROSA: Reconstructing Object Shape and Appearance Textures by Adaptive Detail Transfer](https://mlsanity.com/paper/2501.18595/) (2025)
- [Radiometrically Consistent Gaussian Surfels for Inverse Rendering](https://mlsanity.com/paper/2603.01491/) (2026)
- [SplitNeRF: Split Sum Approximation Neural Field for Joint Geometry, Illumination, and Material Estimation](https://mlsanity.com/paper/2311.16671/) (2023)
- [Multi-times Monte Carlo Rendering for Inter-reflection Reconstruction](https://mlsanity.com/paper/2407.05771/) (2024)
- [UniVoxel: Fast Inverse Rendering by Unified Voxelization of Scene Representation](https://mlsanity.com/paper/2407.19542/) (2024)
- [Neural Inverse Rendering from Propagating Light](https://mlsanity.com/paper/2506.05347/) (2025)

## Abstract (arXiv)

Inverse rendering methods aim to estimate geometry, materials and illumination from multi-view RGB images. In order to achieve better decomposition, recent approaches attempt to model indirect illuminations reflected from different materials via Spherical Gaussians (SG), which, however, tends to blur the high-frequency reflection details. In this paper, we propose an end-to-end inverse rendering pipeline that decomposes materials and illumination from multi-view images, while considering near-field indirect illumination. In a nutshell, we introduce the Monte Carlo sampling based path tracing and cache the indirect illumination as neural radiance, enabling a physics-faithful and easy-to-optimize inverse rendering method. To enhance efficiency and practicality, we leverage SG to represent the smooth environment illuminations and apply importance sampling techniques. To supervise indirect illuminations from unobserved directions, we develop a novel radiance consistency constraint between implicit neural radiance and path tracing results of unobserved rays along with the joint optimization of materials and illuminations, thus significantly improving the decomposition performance. Extensive experiments demonstrate that our method outperforms the state-of-the-art on multiple synthetic and real datasets, especially in terms of inter-reflection decomposition.Our code and data are available at https://woolseyyy.github.io/nefii/.