mlsanityData to 14 Oct 2026

Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition

arXiv 2110.14373 ↗First posted 27 Oct 2021cs.CV cs.GR cs.LGcited by 205

Not on the map. Why?

Abstract (arXiv)

Decomposing a scene into its shape, reflectance and illumination is a fundamental problem in computer vision and graphics. Neural approaches such as NeRF have achieved remarkable success in view synthesis, but do not explicitly perform decomposition and instead operate exclusively on radiance (the product of reflectance and illumination). Extensions to NeRF, such as NeRD, can perform decomposition but struggle to accurately recover detailed illumination, thereby significantly limiting realism. We propose a novel reflectance decomposition network that can estimate shape, BRDF, and per-image illumination given a set of object images captured under varying illumination. Our key technique is a novel illumination integration network called Neural-PIL that replaces a costly illumination integral operation in the rendering with a simple network query. In addition, we also learn deep low-dimensional priors on BRDF and illumination representations using novel smooth manifold auto-encoders. Our decompositions can result in considerably better BRDF and light estimates enabling more accurate novel view-synthesis and relighting compared to prior art. Project page: https://markboss.me/publication/2021-neural-pil/

Key facts. Cited by 205 papers in the map (data through 14 Oct 2026). Builds on 38 of its 66 references that are in the map. First posted on arXiv 27 Oct 2021.

Builds on

38 of 66 references.

References · 38

  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  2. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  3. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  5. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
  6. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  7. 2016Least Squares Generative Adversarial Networkscited by 1,001
  8. 2020Neural Sparse Voxel Fieldscited by 959
  9. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 905
  10. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  11. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  12. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 496
26 more
  1. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 468
  2. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 449
  3. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 335
  4. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
  5. 2019Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Imagecited by 203
  6. 2020Neural Reflectance Fields for Appearance Acquisitioncited by 196
  7. 2017Learning to Predict Indoor Illumination from a Single Imagecited by 161
  8. 2018Single-Image SVBRDF Capture with a Rendering-Aware Deep Networkcited by 116
  9. 2021Neural Lumigraph Renderingcited by 105
  10. 2019Neural Inverse Rendering of an Indoor Scene from a Single Imagecited by 101
  11. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  12. 2020Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Imagescited by 98
  13. 2018Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizercited by 88
  14. 2019Deep Parametric Indoor Lighting Estimationcited by 83
  15. 2018Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Imagecited by 66
  16. 2020Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Imagescited by 65
  17. 2020Two-shot Spatially-varying BRDF and Shape Estimationcited by 59
  18. 2019Neural Illumination: Lighting Prediction for Indoor Environmentscited by 58
  19. 2018Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networkscited by 51
  20. 2019Flexible SVBRDF Capture with a Multi-Image Deep Networkcited by 50
  21. 2021Generative Modelling of BRDF Textures from Flash Imagescited by 35
  22. 2020Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfacescited by 33
  23. 2020Guided Fine-Tuning for Large-Scale Material Transfercited by 27
  24. 2018Learning to Estimate Indoor Lighting from 3D Objectscited by 17
  25. 2020Invertible Neural BRDF for Object Inverse Renderingcited by 17
  26. 2020Deep SVBRDF Estimation on Real Materialscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 34; 2023: 66; 2024: 54; 2025: 30; 2026: 18.

205 papers.

Sort:

By area:

  1. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 575
  2. 2021Neural Fields in Visual Computing and Beyondcited by 540
  3. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  4. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  5. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
  6. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 297
  7. 2023GS-IR: 3D Gaussian Splatting for Inverse Renderingcited by 224
  8. 2023TensoIR: Tensorial Inverse Renderingcited by 168
  9. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  10. 2023NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Imagescited by 163
  11. 2022NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fieldscited by 152
  12. 2023NeRF-DS: Neural Radiance Fields for Dynamic Specular Objectscited by 111
38 more
  1. 2022NeILF: Neural Incident Light Field for Physically-based Material Estimationcited by 105
  2. 2023SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Fieldcited by 94
  3. 2024Neural Gaffer: Relighting Any Object via Diffusioncited by 91
  4. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 90
  5. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
  6. 2023GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorizationcited by 87
  7. 2022Neural 3D Reconstruction in the Wildcited by 79
  8. 2023NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimationcited by 74
  9. 2022PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereocited by 69
  10. 2022DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Imagescited by 69
  11. 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 66
  12. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  13. 2023ENVIDR: Implicit Differentiable Renderer with Neural Environment Lightingcited by 65
  14. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  15. 2022Interactive Segmentation of Radiance Fieldscited by 60
  16. 2023Factor Fields: A Unified Framework for Neural Fields and Beyondcited by 58
  17. 2023NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotationscited by 53
  18. 2022PANDORA: Polarization-Aided Neural Decomposition Of Radiancecited by 52
  19. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51
  20. 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Modelscited by 50
  21. 2022PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fieldscited by 45
  22. 2023Neural-PBIR Reconstruction of Shape, Material, and Illuminationcited by 45
  23. 2023I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFscited by 42
  24. 2023Relighting Neural Radiance Fields with Shadow and Highlight Hintscited by 42
  25. 2025SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Imagescited by 42
  26. 2022Neural Radiance Transfer Fields for Relightable Novel-view Synthesis with Global Illuminationcited by 41
  27. 2023FLARE: Fast Learning of Animatable and Relightable Mesh Avatarscited by 39
  28. 2023NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illuminationcited by 38
  29. 2023Neural Microfacet Fields for Inverse Renderingcited by 38
  30. 2022VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relightingcited by 37
  31. 2022ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Fieldcited by 37
  32. 2024Gaussian Splashing: Unified Particles for Versatile Motion Synthesis and Renderingcited by 37
  33. 2024A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesiscited by 37
  34. 2022NeROIC: Neural Rendering of Objects from Online Image Collectionscited by 36
  35. 2024MeshAvatar: Learning High-quality Triangular Human Avatars from Multi-view Videoscited by 34
  36. 2022Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemblecited by 31
  37. 2023OpenIllumination: A Multi-Illumination Dataset for Inverse Rendering Evaluation on Real Objectscited by 31
  38. 2022S3-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpointcited by 30

50 most cited of 205 All 205 (JSON)