mlsanityData to 7 Oct 2026

Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image

arXiv 1905.02722 ↗First posted 7 May 2019cs.CVcited by 202

Not on the map. Why?

Cited by 26 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a deep inverse rendering framework for indoor scenes. From a single RGB image of an arbitrary indoor scene, we create a complete scene reconstruction, estimating shape, spatially-varying lighting, and spatially-varying, non-Lambertian surface reflectance. To train this network, we augment the SUNCG indoor scene dataset with real-world materials and render them with a fast, high-quality, physically-based GPU renderer to create a large-scale, photorealistic indoor dataset. Our inverse rendering network incorporates physical insights -- including a spatially-varying spherical Gaussian lighting representation, a differentiable rendering layer to model scene appearance, a cascade structure to iteratively refine the predictions and a bilateral solver for refinement -- allowing us to jointly reason about shape, lighting, and reflectance. Experiments show that our framework outperforms previous methods for estimating individual scene components, which also enables various novel applications for augmented reality, such as photorealistic object insertion and material editing. Code and data will be made publicly available.

Key facts. Cited by 202 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 31; 2023: 53; 2024: 33; 2025: 39; 2026: 21.

202 papers.

Sort:

By area:

  1. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  2. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  3. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
  4. 2021Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decompositioncited by 204
  5. 2022Modeling Indirect Illumination for Inverse Renderingcited by 167
  6. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  7. 2023NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Imagescited by 163
  8. 2024RGBleftrightarrowX: Image decomposition and synthesis using material- and lighting-aware diffusion modelscited by 159
  9. 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illuminationcited by 135
  10. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
  11. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 89
  12. 2021Learning Indoor Inverse Rendering with 3D Spatially-Varying Lightingcited by 82
38 more
  1. 2022Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracingcited by 78
  2. 2025Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysiscited by 78
  3. 2023Intrinsic Image Diffusion for Indoor Single-view Material Estimationcited by 73
  4. 2021Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Renderingcited by 71
  5. 2022Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptationcited by 67
  6. 2022Physically-Based Editing of Indoor Scene Lighting from a Single Imagecited by 62
  7. 2022IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenescited by 62
  8. 2025DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Modelscited by 55
  9. 2021Shape from Polarization for Complex Scenes in the Wildcited by 53
  10. 2023Intrinsic Image Decomposition via Ordinal Shadingcited by 53
  11. 2021DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderercited by 52
  12. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51
  13. 2021Free-viewpoint Indoor Neural Relighting from Multi-view Stereocited by 50
  14. 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Modelscited by 50
  15. 2024Colorful Diffuse Intrinsic Image Decomposition in the Wildcited by 44
  16. 2023I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFscited by 42
  17. 2024MatSynth: A Modern PBR Materials Datasetcited by 42
  18. 2023NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illuminationcited by 38
  19. 2025UniRelight: Learning Joint Decomposition and Synthesis for Video Relightingcited by 38
  20. 2021A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyondcited by 37
  21. 2021Shape and Material Capture at Homecited by 36
  22. 2025LightLab: Controlling Light Sources in Images with Diffusion Modelscited by 36
  23. 2022Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertioncited by 35
  24. 2023NeMF: Inverse Volume Rendering with Neural Microflake Fieldcited by 34
  25. 2023EverLight: Indoor-Outdoor Editable HDR Lighting Estimationcited by 32
  26. 2022De-rendering 3D Objects in the Wildcited by 31
  27. 2022OutCast: Outdoor Single-image Relighting with Cast Shadowscited by 30
  28. 2023SpecNeRF: Gaussian Directional Encoding for Specular Reflectionscited by 30
  29. 2024IllumiNeRF: 3D Relighting Without Inverse Renderingcited by 29
  30. 2021GMLight: Lighting Estimation via Geometric Distribution Approximationcited by 28
  31. 2021De-rendering the World's Revolutionary Artefactscited by 28
  32. 2023Material Palette: Extraction of Materials from a Single Imagecited by 28
  33. 2024MaPa: Text-driven Photorealistic Material Painting for 3D Shapescited by 27
  34. 2024Photorealistic Object Insertion with Diffusion-Guided Inverse Renderingcited by 27
  35. 2022PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decompositioncited by 26
  36. 2023Materialistic: Selecting Similar Materials in Imagescited by 26
  37. 2023NeISF: Neural Incident Stokes Field for Geometry and Material Estimationcited by 25
  38. 2022Multi-view Inverse Rendering for Large-scale Real-world Indoor Scenescited by 23

50 most cited of 202 All 202 (JSON)