# Unsupervised Deep Single-Image Intrinsic Decomposition using Illumination-Varying Image Sequences

arXiv: https://arxiv.org/abs/1803.00805 · HTML: https://mlsanity.com/paper/1803.00805/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2018.

## Built on by (most cited first, 22 of 22)

- [Intrinsic Image Decomposition via Ordinal Shading](https://mlsanity.com/paper/2311.12792/) (2023)
- [DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer](https://mlsanity.com/paper/2111.00140/) (2021)
- [Colorful Diffuse Intrinsic Image Decomposition in the Wild](https://mlsanity.com/paper/2409.13690/) (2024)
- [PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decomposition](https://mlsanity.com/paper/2203.16670/) (2022)
- [Estimating Reflectance Layer from A Single Image: Integrating Reflectance Guidance and Shadow/Specular Aware Learning](https://mlsanity.com/paper/2211.14751/) (2022)
- [LightDepth: Single-View Depth Self-Supervision from Illumination Decline](https://mlsanity.com/paper/2308.10525/) (2023)
- [Unsupervised Intrinsic Image Decomposition with LiDAR Intensity](https://mlsanity.com/paper/2303.10820/) (2023)
- [Relighting from a Single Image: Datasets and Deep Intrinsic-based Architecture](https://mlsanity.com/paper/2409.18770/) (2024)
- [UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors](https://mlsanity.com/paper/2605.00658/) (2026)
- [Memory-guided Image De-raining Using Time-Lapse Data](https://mlsanity.com/paper/2201.01883/) (2022)
- [Discriminative feature encoding for intrinsic image decomposition](https://mlsanity.com/paper/2209.12155/) (2022)
- [MLI-NeRF: Multi-Light Intrinsic-Aware Neural Radiance Fields](https://mlsanity.com/paper/2411.17235/) (2024)
- [FlowIID: Single-Step Intrinsic Image Decomposition via Latent Flow Matching](https://mlsanity.com/paper/2601.12329/) (2026)
- [An Optical physics inspired CNN approach for intrinsic image decomposition](https://mlsanity.com/paper/2105.10076/) (2021)
- [Self-Supervised Intrinsic Image Decomposition Network Considering Reflectance Consistency](https://mlsanity.com/paper/2111.04506/) (2021)
- [Learning Intrinsic Images for Clothing](https://mlsanity.com/paper/2111.08521/) (2021)
- [Face Inverse Rendering via Hierarchical Decoupling](https://mlsanity.com/paper/2301.06733/) (2023)
- [Unsupervised Intrinsic Image Decomposition with LiDAR Intensity Enhanced Training](https://mlsanity.com/paper/2403.14089/) (2024)
- [Learning Relighting and Intrinsic Decomposition in Neural Radiance Fields](https://mlsanity.com/paper/2406.11077/) (2024)
- [A Prototype Unit for Image De-raining using Time-Lapse Data](https://mlsanity.com/paper/2412.19459/) (2024)
- [Clair Obscur: an Illumination-Aware Method for Real-World Image Vectorization](https://mlsanity.com/paper/2511.20034/) (2025)
- [LumiX: Structured and Coherent Text-to-Intrinsic Generation](https://mlsanity.com/paper/2512.02781/) (2025)

## Abstract (arXiv)

Machine learning based Single Image Intrinsic Decomposition (SIID) methods decompose a captured scene into its albedo and shading images by using the knowledge of a large set of known and realistic ground truth decompositions. Collecting and annotating such a dataset is an approach that cannot scale to sufficient variety and realism. We free ourselves from this limitation by training on unannotated images. Our method leverages the observation that two images of the same scene but with different lighting provide useful information on their intrinsic properties: by definition, albedo is invariant to lighting conditions, and cross-combining the estimated albedo of a first image with the estimated shading of a second one should lead back to the second one's input image. We transcribe this relationship into a siamese training scheme for a deep convolutional neural network that decomposes a single image into albedo and shading. The siamese setting allows us to introduce a new loss function including such cross-combinations, and to train solely on (time-lapse) images, discarding the need for any ground truth annotations. As a result, our method has the good properties of i) taking advantage of the time-varying information of image sequences in the (pre-computed) training step, ii) not requiring ground truth data to train on, and iii) being able to decompose single images of unseen scenes at runtime. To demonstrate and evaluate our work, we additionally propose a new rendered dataset containing illumination-varying scenes and a set of quantitative metrics to evaluate SIID algorithms. Despite its unsupervised nature, our results compete with state of the art methods, including supervised and non data-driven methods.