mlsanityData to 7 Oct 2026

Colorful Diffuse Intrinsic Image Decomposition in the Wild

arXiv 2409.13690 ↗First posted 20 Sep 2024cs.CVcited by 44

Not on the map. Why?

Abstract (arXiv)

Intrinsic image decomposition aims to separate the surface reflectance and the effects from the illumination given a single photograph. Due to the complexity of the problem, most prior works assume a single-color illumination and a Lambertian world, which limits their use in illumination-aware image editing applications. In this work, we separate an input image into its diffuse albedo, colorful diffuse shading, and specular residual components. We arrive at our result by gradually removing first the single-color illumination and then the Lambertian-world assumptions. We show that by dividing the problem into easier sub-problems, in-the-wild colorful diffuse shading estimation can be achieved despite the limited ground-truth datasets. Our extended intrinsic model enables illumination-aware analysis of photographs and can be used for image editing applications such as specularity removal and per-pixel white balancing.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 31 of its 53 references that are in the map. First posted on arXiv 20 Sep 2024.

Builds on

31 of 53 references.

References · 31

  1. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  2. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  3. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  4. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
  5. 2019Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Imagecited by 202
  6. 2023MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyondcited by 185
  7. 2024RGBleftrightarrowX: Image decomposition and synthesis using material- and lighting-aware diffusion modelscited by 159
  8. 2019Neural Inverse Rendering of an Indoor Scene from a Single Imagecited by 101
  9. 2018CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Renderingcited by 85
  10. 2021Learning Indoor Inverse Rendering with 3D Spatially-Varying Lightingcited by 82
  11. 2022Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracingcited by 78
  12. 2018Learning Intrinsic Image Decomposition from Watching the Worldcited by 74
19 more
  1. 2022Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptationcited by 67
  2. 2024IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illuminationcited by 67
  3. 2022IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenescited by 62
  4. 2019Unsupervised Learning for Intrinsic Image Decomposition from a Single Imagecited by 55
  5. 2023Intrinsic Image Decomposition via Ordinal Shadingcited by 53
  6. 2017Self-Supervised Intrinsic Image Decompositioncited by 43
  7. 2016Learning Non-Lambertian Object Intrinsics across ShapeNet Categoriescited by 42
  8. 2021A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyondcited by 37
  9. 2015Convolutional Color Constancycited by 36
  10. 2019Automatic Scene Inference for 3D Object Compositingcited by 36
  11. 2018LIME: Live Intrinsic Material Estimationcited by 33
  12. 2020EDEN: Multimodal Synthetic Dataset of Enclosed GarDEN Scenescited by 29
  13. 2018Joint Learning of Intrinsic Images and Semantic Segmentationcited by 26
  14. 2022PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decompositioncited by 26
  15. 2018Unsupervised Deep Single-Image Intrinsic Decomposition using Illumination-Varying Image Sequencescited by 22
  16. 2017CNN based Learning using Reflection and Retinex Models for Intrinsic Image Decompositioncited by 19
  17. 2019Real-Time Global Illumination Decomposition of Videoscited by 18
  18. 2023Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluationcited by 16
  19. 2024Scale-Invariant Monocular Depth Estimation via SSI Depthcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 21; 2026: 21.

44 papers.

Sort:

By area:

  1. 2025Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysiscited by 78
  2. 2024LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relightingcited by 37
  3. 2024ScribbleLight: Single Image Indoor Relighting with Scribblescited by 22
  4. 2025IntrinsiX: High-Quality PBR Generation using Image Priorscited by 18
  5. 2025Ouroboros: Single-step Diffusion Models for Cycle-consistent Forward and Inverse Renderingcited by 16
  6. 2025Physically Controllable Relighting of Photographscited by 14
  7. 2025DNF-Intrinsic: Deterministic Noise-Free Diffusion for Indoor Inverse Renderingcited by 11
  8. 2025PRISM: A Unified Framework for Photorealistic Reconstruction and Intrinsic Scene Modelingcited by 9
  9. 2026Generative World Renderercited by 8
  10. 2025PractiLight: Practical Light Control Using Foundational Diffusion Modelscited by 7
  11. 2025Jodi: Unification of Visual Generation and Understanding via Joint Modelingcited by 5
  12. 2025Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Imagescited by 5
32 more
  1. 2025FlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacementcited by 5
  2. 2025MVInverse: Feed-forward Multi-view Inverse Rendering in Secondscited by 5
  3. 2026LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction under Multi-illumination Conditionscited by 4
  4. 2025ePBR: Extended PBR Materials in Image Synthesiscited by 3
  5. 2026UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priorscited by 3
  6. 2025StableMTL: Repurposing Latent Diffusion Models for Multi-Task Learning from Partially Annotated Synthetic Datasetscited by 2
  7. 2026SeeClear: Reliable Transparent Object Depth Estimation via Generative Opacificationcited by 2
  8. 2025Generative Perception of Shape and Material from Differential Motioncited by 1
  9. 2025From Gallery to Wrist: Realistic 3D Bracelet Insertion in Videoscited by 1
  10. 2025MVID: Feed-Forward Multi-View Intrinsic Image Decompositioncited by 1
  11. 2026FlowIID: Single-Step Intrinsic Image Decomposition via Latent Flow Matchingcited by 1
  12. 2026Workflow-Aware Structured Layer Decomposition for Illustration Productioncited by 1
  13. 2026SGS-Intrinsic: Semantic-Invariant Gaussian Splatting for Sparse-View Indoor Inverse Renderingcited by 1
  14. 2025Texture-aware Intrinsic Image Decomposition with Model- and Learning-based Priorscited by 0
  15. 2025VT-Intrinsic: Physics-Based Decomposition of Reflectance and Shading using a Single Visible-Thermal Image Paircited by 0
  16. 2025IPCD: Intrinsic Point-Cloud Decompositioncited by 0
  17. 2025Clair Obscur: an Illumination-Aware Method for Real-World Image Vectorizationcited by 0
  18. 2025LumiX: Structured and Coherent Text-to-Intrinsic Generationcited by 0
  19. 2026Relighting as a Probe of Visual Priors via Augmented Latent Intrinsicscited by 0
  20. 2026Cycle-Consistent Tuning for Layered Image Decompositioncited by 0
  21. 2026Olbedo: An Albedo and Shading Aerial Dataset for Large-Scale Outdoor Environmentscited by 0
  22. 2026Geo-ID: Test-Time Geometric Consensus for Cross-View Consistent Intrinsicscited by 0
  23. 2026Under One Sun: Multi-Object Generative Perception of Materials and Illuminationcited by 0
  24. 2026Rethinking Exposure Correction for Spatially Non-uniform Degradationcited by 0
  25. 2026PhysEditBench: A Protocol-Conditioned Benchmark for Dense Physical-Map Prediction with Image Editorscited by 0
  26. 2026ResEdit: Residual embeddings for precise generative image editingcited by 0
  27. 2026DANTE-W: Diffuse Albedo Neural Texturing in the Wildcited by 0
  28. 2026Intrinsic decomposition and editing of 3D Gaussian splatscited by 0
  29. 2026FreeLit: Paired-Free Indoor Relighting via Physics-Guided Diffusioncited by 0
  30. 2026Unified Video Dense Prediction from Disjoint Datacited by 0
  31. 2026Dior: Drawing the Light of Image via Material-Decoupled Illumination Representationcited by 0
  32. 2026EvenSplat: Coupled 2D-3D Decomposition for Gaussian Splatting under Exposure and Illumination Variationcited by 0