mlsanityData to 14 Oct 2026

InverseRenderNet: Learning single image inverse rendering

arXiv 1811.12328 ↗First posted 29 Nov 2018cs.CVcited by 100

Not on the map. Why?

Abstract (arXiv)

We show how to train a fully convolutional neural network to perform inverse rendering from a single, uncontrolled image. The network takes an RGB image as input, regresses albedo and normal maps from which we compute lighting coefficients. Our network is trained using large uncontrolled image collections without ground truth. By incorporating a differentiable renderer, our network can learn from self-supervision. Since the problem is ill-posed we introduce additional supervision: 1. We learn a statistical natural illumination prior, 2. Our key insight is to perform offline multiview stereo (MVS) on images containing rich illumination variation. From the MVS pose and depth maps, we can cross project between overlapping views such that Siamese training can be used to ensure consistent estimation of photometric invariants. MVS depth also provides direct coarse supervision for normal map estimation. We believe this is the first attempt to use MVS supervision for learning inverse rendering.

Key facts. Cited by 100 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 23; 2022: 17; 2023: 22; 2024: 20; 2025: 14; 2026: 4.

100 papers.

Sort:

By area:

  1. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 286
  2. 2022Modeling Indirect Illumination for Inverse Renderingcited by 168
  3. 2023NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Imagescited by 163
  4. 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illuminationcited by 135
  5. 2021NeRF for Outdoor Scene Relightingcited by 114
  6. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 90
  7. 2021Learning Indoor Inverse Rendering with 3D Spatially-Varying Lightingcited by 83
  8. 2024IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illuminationcited by 67
  9. 2025DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Modelscited by 57
  10. 2021Shape from Polarization for Complex Scenes in the Wildcited by 53
  11. 2021DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderercited by 52
  12. 2022PANDORA: Polarization-Aided Neural Decomposition Of Radiancecited by 52
38 more
  1. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51
  2. 2022IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesiscited by 50
  3. 2023Generative Models: What Do They Know? Do They Know Things? Let's Find Out!cited by 50
  4. 2021AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANscited by 49
  5. 2021Self-supervised Outdoor Scene Relightingcited by 48
  6. 2023StyleGAN knows Normal, Depth, Albedo, and Morecited by 44
  7. 2022Relighting4D: Neural Relightable Human from Videoscited by 41
  8. 2025UniRelight: Learning Joint Decomposition and Synthesis for Video Relightingcited by 40
  9. 2023NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illuminationcited by 38
  10. 2021A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyondcited by 37
  11. 2021Shape and Material Capture at Homecited by 36
  12. 2023NeMF: Inverse Volume Rendering with Neural Microflake Fieldcited by 34
  13. 2022De-rendering 3D Objects in the Wildcited by 31
  14. 2021De-rendering the World's Revolutionary Artefactscited by 28
  15. 2024Photorealistic Object Insertion with Diffusion-Guided Inverse Renderingcited by 27
  16. 2022PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decompositioncited by 26
  17. 2023UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Videocited by 22
  18. 2023IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracingcited by 20
  19. 2023LightSim: Neural Lighting Simulation for Urban Scenescited by 20
  20. 2024GenLit: Reformulating Single-Image Relighting as Video Generationcited by 20
  21. 2022SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Mapscited by 19
  22. 2023SIRe-IR: Inverse Rendering for BRDF Reconstruction with Shadow and Illumination Removal in High-Illuminance Scenescited by 19
  23. 2024Neural LightRig: Unlocking Accurate Object Normal and Material Estimation with Multi-Light Diffusioncited by 19
  24. 2021Outdoor inverse rendering from a single image using multiview self-supervisioncited by 18
  25. 2021Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Visioncited by 18
  26. 2022Estimating Reflectance Layer from A Single Image: Integrating Reflectance Guidance and Shadow/Specular Aware Learningcited by 18
  27. 2024PRTGS: Precomputed Radiance Transfer of Gaussian Splats for Real-Time High-Quality Relightingcited by 18
  28. 2022SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Imagescited by 17
  29. 2024Latent Intrinsics Emerge from Training to Relightcited by 17
  30. 2023Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluationcited by 16
  31. 2025LuxDiT: Lighting Estimation with Video Diffusion Transformercited by 15
  32. 2021Spatially-Varying Outdoor Lighting Estimation from Intrinsicscited by 14
  33. 2023Weakly-supervised Single-view Image Relightingcited by 14
  34. 2025Rethinking Driving World Model as Synthetic Data Generator for Perception Taskscited by 14
  35. 2023Accidental Light Probescited by 13
  36. 2024PBR-NeRF: Inverse Rendering with Physics-Based Neural Fieldscited by 13
  37. 2024Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Videocited by 12
  38. 2021Deep Image-based Illumination Harmonizationcited by 11

50 most cited of 100 All 100 (JSON)