# PBR-NeRF: Inverse Rendering with Physics-Based Neural Fields

arXiv: https://arxiv.org/abs/2412.09680 · HTML: https://mlsanity.com/paper/2412.09680/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 56 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 30 of 38)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Modeling Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2204.06837/) (2022)
- [TensoIR: Tensorial Inverse Rendering](https://mlsanity.com/paper/2304.12461/) (2023)
- [NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images](https://mlsanity.com/paper/2305.17398/) (2023)
- [NeRF for Outdoor Scene Relighting](https://mlsanity.com/paper/2112.05140/) (2021)
- [NeILF: Neural Incident Light Field for Physically-based Material Estimation](https://mlsanity.com/paper/2203.07182/) (2022)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [InverseRenderNet: Learning single image inverse rendering](https://mlsanity.com/paper/1811.12328/) (2018)
- [EventNeRF: Neural Radiance Fields from a Single Colour Event Camera](https://mlsanity.com/paper/2206.11896/) (2022)
- [Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenes](https://mlsanity.com/paper/2304.03266/) (2023)
- [Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting](https://mlsanity.com/paper/2109.06061/) (2021)
- [Learning Signed Distance Field for Multi-view Surface Reconstruction](https://mlsanity.com/paper/2108.09964/) (2021)
- [NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimation](https://mlsanity.com/paper/2303.17147/) (2023)
- [MuRF: Multi-Baseline Radiance Fields](https://mlsanity.com/paper/2312.04565/) (2023)
- [Neural LiDAR Fields for Novel View Synthesis](https://mlsanity.com/paper/2305.01643/) (2023)

## Built on by (most cited first, 13 of 13)

- [I2-NeRF: Learning Neural Radiance Fields Under Physically-Grounded Media Interactions](https://mlsanity.com/paper/2510.22161/) (2025)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [Generative World Renderer](https://mlsanity.com/paper/2604.02329/) (2026)
- [MatSpray: Fusing 2D Material World Knowledge on 3D Geometry](https://mlsanity.com/paper/2512.18314/) (2025)
- [LidarPainter: One-Step Away From Any Lidar View To Novel Guidance](https://mlsanity.com/paper/2507.12114/) (2025)
- [PS-GS: Gaussian Splatting for Multi-View Photometric Stereo](https://mlsanity.com/paper/2507.18231/) (2025)
- [VIRGi: View-dependent Instant Recoloring of 3D Gaussians Splats](https://mlsanity.com/paper/2603.02986/) (2026)
- [LightMover: Generative Light Movement with Color and Intensity Controls](https://mlsanity.com/paper/2603.27209/) (2026)
- [Physically-Guided Optical Inversion Enable Non-Contact Side-Channel Attack on Isolated Screens](https://mlsanity.com/paper/2604.13419/) (2026)
- [BRDFusion: Physics Meets Generation for Urban Scene Inverse Rendering](https://mlsanity.com/paper/2606.17049/) (2026)
- [Learning Spectral and Polarimetric Clues for One-to-Multimodal Novel View Synthesis](https://mlsanity.com/paper/2607.02372/) (2026)
- [Beyond Thermal Imaging: Inferring Thermophysical Properties from Time-Resolved Thermal Observations](https://mlsanity.com/paper/2607.07962/) (2026)
- [Towards Physically Grounded Articulated 3D Assets: A Multi-Agent Pipeline and Benchmark](https://mlsanity.com/paper/2610.13947/) (2026)

## Abstract (arXiv)

We tackle the ill-posed inverse rendering problem in 3D reconstruction with a Neural Radiance Field (NeRF) approach informed by Physics-Based Rendering (PBR) theory, named PBR-NeRF. Our method addresses a key limitation in most NeRF and 3D Gaussian Splatting approaches: they estimate view-dependent appearance without modeling scene materials and illumination. To address this limitation, we present an inverse rendering (IR) model capable of jointly estimating scene geometry, materials, and illumination. Our model builds upon recent NeRF-based IR approaches, but crucially introduces two novel physics-based priors that better constrain the IR estimation. Our priors are rigorously formulated as intuitive loss terms and achieve state-of-the-art material estimation without compromising novel view synthesis quality. Our method is easily adaptable to other inverse rendering and 3D reconstruction frameworks that require material estimation. We demonstrate the importance of extending current neural rendering approaches to fully model scene properties beyond geometry and view-dependent appearance. Code is publicly available at https://github.com/s3anwu/pbrnerf