# NeRF as a Non-Distant Environment Emitter in Physics-based Inverse Rendering

arXiv: https://arxiv.org/abs/2402.04829 · HTML: https://mlsanity.com/paper/2402.04829/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 59 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images](https://mlsanity.com/paper/2111.13679/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis](https://mlsanity.com/paper/2302.14859/) (2023)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [Neural Reflectance Fields for Appearance Acquisition](https://mlsanity.com/paper/2008.03824/) (2020)
- [Modeling Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2204.06837/) (2022)
- [Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising](https://mlsanity.com/paper/2206.03380/) (2022)
- [IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images](https://mlsanity.com/paper/2204.02232/) (2022)
- [Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images](https://mlsanity.com/paper/2007.09892/) (2020)
- [SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections](https://mlsanity.com/paper/2205.15768/) (2022)
- [VR-NeRF: High-Fidelity Virtualized Walkable Spaces](https://mlsanity.com/paper/2311.02542/) (2023)
- [PermutoSDF: Fast Multi-View Reconstruction with Implicit Surfaces using Permutohedral Lattices](https://mlsanity.com/paper/2211.12562/) (2022)
- [Adaptive Shells for Efficient Neural Radiance Field Rendering](https://mlsanity.com/paper/2311.10091/) (2023)
- [NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illumination](https://mlsanity.com/paper/2303.16617/) (2023)
- [S3-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint](https://mlsanity.com/paper/2210.08936/) (2022)
- [Differentiable Rendering of Neural SDFs through Reparameterization](https://mlsanity.com/paper/2206.05344/) (2022)
- [ShadowNeuS: Neural SDF Reconstruction by Shadow Ray Supervision](https://mlsanity.com/paper/2211.14086/) (2022)
- [Dr.Jit: A Just-In-Time Compiler for Differentiable Rendering](https://mlsanity.com/paper/2202.01284/) (2022)
- [Noise-Aware Merging of High Dynamic Range Image Stacks without Camera Calibration](https://mlsanity.com/paper/2009.07975/) (2020)
- [A Learned Radiance-Field Representation for Complex Luminaires](https://mlsanity.com/paper/2207.05009/) (2022)

## Built on by (most cited first, 8 of 8)

- [Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering](https://mlsanity.com/paper/2409.05867/) (2024)
- [GaSLight: Gaussian Splats for Spatially-Varying Lighting in HDR](https://mlsanity.com/paper/2504.10809/) (2025)
- [GS-ROR2: Bidirectional-guided 3DGS and SDF for Reflective Object Relighting and Reconstruction](https://mlsanity.com/paper/2406.18544/) (2024)
- [Interactive Rendering of Relightable and Animatable Gaussian Avatars](https://mlsanity.com/paper/2407.10707/) (2024)
- [MRD: Using Physically Based Differentiable Rendering to Probe Vision Models for 3D Scene Understanding](https://mlsanity.com/paper/2512.12307/) (2025)
- [Neural Inverse Rendering from Propagating Light](https://mlsanity.com/paper/2506.05347/) (2025)
- [EAG-PT: Emission-Aware Gaussians and Path Tracing for Diffuse Indoor Scene Reconstruction and Editing](https://mlsanity.com/paper/2601.23065/) (2026)
- [VisibilitySplat: Ray-Traced Occlusion for Relighting with 3D Gaussian Splatting](https://mlsanity.com/paper/2610.19847/) (2026)

## Abstract (arXiv)

Physics-based inverse rendering enables joint optimization of shape, material, and lighting based on captured 2D images. To ensure accurate reconstruction, using a light model that closely resembles the captured environment is essential. Although the widely adopted distant environmental lighting model is adequate in many cases, we demonstrate that its inability to capture spatially varying illumination can lead to inaccurate reconstructions in many real-world inverse rendering scenarios. To address this limitation, we incorporate NeRF as a non-distant environment emitter into the inverse rendering pipeline. Additionally, we introduce an emitter importance sampling technique for NeRF to reduce the rendering variance. Through comparisons on both real and synthetic datasets, our results demonstrate that our NeRF-based emitter offers a more precise representation of scene lighting, thereby improving the accuracy of inverse rendering.