mlsanityData to 14 Oct 2026

Neural Inverse Rendering of an Indoor Scene from a Single Image

arXiv 1901.02453 ↗First posted 8 Jan 2019cs.CVcited by 101

Not on the map. Why?

Abstract (arXiv)

Inverse rendering aims to estimate physical attributes of a scene, e.g., reflectance, geometry, and lighting, from image(s). Inverse rendering has been studied primarily for single objects or with methods that solve for only one of the scene attributes. We propose the first learning-based approach that jointly estimates albedo, normals, and lighting of an indoor scene from a single image. Our key contribution is the Residual Appearance Renderer (RAR), which can be trained to synthesize complex appearance effects (e.g., inter-reflection, cast shadows, near-field illumination, and realistic shading), which would be neglected otherwise. This enables us to perform self-supervised learning on real data using a reconstruction loss, based on re-synthesizing the input image from the estimated components. We finetune with real data after pretraining with synthetic data. To this end, we use physically-based rendering to create a large-scale synthetic dataset, which is a significant improvement over prior datasets. Experimental results show that our approach outperforms state-of-the-art methods that estimate one or more scene attributes.

Key facts. Cited by 101 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 15; 2022: 21; 2023: 23; 2024: 18; 2025: 17; 2026: 7.

101 papers.

Sort:

By area:

  1. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 286
  2. 2021Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decompositioncited by 205
  3. 2022Modeling Indirect Illumination for Inverse Renderingcited by 168
  4. 2023Infinite Photorealistic Worlds using Procedural Generationcited by 147
  5. 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illuminationcited by 135
  6. 2021NeRF for Outdoor Scene Relightingcited by 114
  7. 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
  8. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 90
  9. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
  10. 2021Learning Indoor Inverse Rendering with 3D Spatially-Varying Lightingcited by 83
  11. 2022Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracingcited by 78
  12. 2022DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Imagescited by 69
38 more
  1. 2022Physically-Based Editing of Indoor Scene Lighting from a Single Imagecited by 62
  2. 2022IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenescited by 62
  3. 2025DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Modelscited by 57
  4. 2023Intrinsic Image Decomposition via Ordinal Shadingcited by 53
  5. 2021Free-viewpoint Indoor Neural Relighting from Multi-view Stereocited by 50
  6. 2021Self-supervised Outdoor Scene Relightingcited by 48
  7. 2022Diverse Plausible 360-Degree Image Outpainting for Efficient 3DCG Background Creationcited by 47
  8. 2024Colorful Diffuse Intrinsic Image Decomposition in the Wildcited by 44
  9. 2022Relighting4D: Neural Relightable Human from Videoscited by 41
  10. 2025UniRelight: Learning Joint Decomposition and Synthesis for Video Relightingcited by 40
  11. 2023NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illuminationcited by 38
  12. 2021A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyondcited by 37
  13. 2021Shape and Material Capture at Homecited by 36
  14. 2022Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertioncited by 35
  15. 2023NeMF: Inverse Volume Rendering with Neural Microflake Fieldcited by 34
  16. 2023Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generationcited by 33
  17. 2023EverLight: Indoor-Outdoor Editable HDR Lighting Estimationcited by 32
  18. 2021GMLight: Lighting Estimation via Geometric Distribution Approximationcited by 28
  19. 2023Material Palette: Extraction of Materials from a Single Imagecited by 28
  20. 2024Photorealistic Object Insertion with Diffusion-Guided Inverse Renderingcited by 27
  21. 2022PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decompositioncited by 26
  22. 2023NeISF: Neural Incident Stokes Field for Geometry and Material Estimationcited by 25
  23. 2022Multi-view Inverse Rendering for Large-scale Real-world Indoor Scenescited by 24
  24. 2024ScribbleLight: Single Image Indoor Relighting with Scribblescited by 24
  25. 2023Factorized Inverse Path Tracing for Efficient and Accurate Material-Lighting Estimationcited by 23
  26. 2022PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenescited by 22
  27. 2023UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Videocited by 22
  28. 2023IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracingcited by 20
  29. 2023LightSim: Neural Lighting Simulation for Urban Scenescited by 20
  30. 2024GenLit: Reformulating Single-Image Relighting as Video Generationcited by 20
  31. 2023SIRe-IR: Inverse Rendering for BRDF Reconstruction with Shadow and Illumination Removal in High-Illuminance Scenescited by 19
  32. 2021Outdoor inverse rendering from a single image using multiview self-supervisioncited by 18
  33. 2024SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wildcited by 18
  34. 2023Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluationcited by 16
  35. 2025LuxDiT: Lighting Estimation with Video Diffusion Transformercited by 15
  36. 2025Physically Controllable Relighting of Photographscited by 14
  37. 2023Accidental Light Probescited by 13
  38. 2024PBR-NeRF: Inverse Rendering with Physics-Based Neural Fieldscited by 13

50 most cited of 101 All 101 (JSON)