# Self-Supervised Intrinsic Image Decomposition

arXiv: https://arxiv.org/abs/1711.03678 · HTML: https://mlsanity.com/paper/1711.03678/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2017.

## Built on by (most cited first, 30 of 43)

- [Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark](https://mlsanity.com/paper/2310.16044/) (2023)
- [Intrinsic Image Decomposition via Ordinal Shading](https://mlsanity.com/paper/2311.12792/) (2023)
- [IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis](https://mlsanity.com/paper/2210.00647/) (2022)
- [Colorful Diffuse Intrinsic Image Decomposition in the Wild](https://mlsanity.com/paper/2409.13690/) (2024)
- [StyleGAN knows Normal, Depth, Albedo, and More](https://mlsanity.com/paper/2306.00987/) (2023)
- [A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyond](https://mlsanity.com/paper/2112.03842/) (2021)
- [Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Dataset](https://mlsanity.com/paper/2504.08541/) (2025)
- [De-rendering 3D Objects in the Wild](https://mlsanity.com/paper/2201.02279/) (2022)
- [De-rendering the World's Revolutionary Artefacts](https://mlsanity.com/paper/2104.03954/) (2021)
- [PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decomposition](https://mlsanity.com/paper/2203.16670/) (2022)
- [ScribbleLight: Single Image Indoor Relighting with Scribbles](https://mlsanity.com/paper/2411.17696/) (2024)
- [SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps](https://mlsanity.com/paper/2201.05120/) (2022)
- [Outdoor inverse rendering from a single image using multiview self-supervision](https://mlsanity.com/paper/2102.06591/) (2021)
- [Latent Intrinsics Emerge from Training to Relight](https://mlsanity.com/paper/2405.21074/) (2024)
- [Weakly-supervised Single-view Image Relighting](https://mlsanity.com/paper/2303.13852/) (2023)
- [Deep Feedback Inverse Problem Solver](https://mlsanity.com/paper/2101.07719/) (2021)
- [Diffusion Reflectance Map: Single-Image Stochastic Inverse Rendering of Illumination and Reflectance](https://mlsanity.com/paper/2312.04529/) (2023)
- [Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theory](https://mlsanity.com/paper/2407.20785/) (2024)
- [Unsupervised Intrinsic Image Decomposition with LiDAR Intensity](https://mlsanity.com/paper/2303.10820/) (2023)
- [Photo-realistic Neural Domain Randomization](https://mlsanity.com/paper/2210.12682/) (2022)
- [Coherent, super resolved radar beamforming using self-supervised learning](https://mlsanity.com/paper/2106.13085/) (2021)
- [StyLitGAN: Prompting StyleGAN to Produce New Illumination Conditions](https://mlsanity.com/paper/2205.10351/) (2022)
- [Joint Learning of Portrait Intrinsic Decomposition and Relighting](https://mlsanity.com/paper/2106.15305/) (2021)
- [A Novel Intrinsic Image Decomposition Method to Recover Albedo for Aerial Images in Photogrammetry Processing](https://mlsanity.com/paper/2204.04142/) (2022)
- [Creating a Forensic Database of Shoeprints from Online Shoe Tread Photos](https://mlsanity.com/paper/2205.02361/) (2022)
- [Physics-based Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2212.04705/) (2022)
- [DPF: Learning Dense Prediction Fields with Weak Supervision](https://mlsanity.com/paper/2303.16890/) (2023)
- [Does resistance to style-transfer equal Global Shape Bias? Measuring network sensitivity to global shape configuration](https://mlsanity.com/paper/2310.07555/) (2023)
- [Learning to Predict Scene-Level Implicit 3D from Posed RGBD Data](https://mlsanity.com/paper/2306.08671/) (2023)
- [Tree-Structured Shading Decomposition](https://mlsanity.com/paper/2309.07122/) (2023)

## Abstract (arXiv)

Intrinsic decomposition from a single image is a highly challenging task, due to its inherent ambiguity and the scarcity of training data. In contrast to traditional fully supervised learning approaches, in this paper we propose learning intrinsic image decomposition by explaining the input image. Our model, the Rendered Intrinsics Network (RIN), joins together an image decomposition pipeline, which predicts reflectance, shape, and lighting conditions given a single image, with a recombination function, a learned shading model used to recompose the original input based off of intrinsic image predictions. Our network can then use unsupervised reconstruction error as an additional signal to improve its intermediate representations. This allows large-scale unlabeled data to be useful during training, and also enables transferring learned knowledge to images of unseen object categories, lighting conditions, and shapes. Extensive experiments demonstrate that our method performs well on both intrinsic image decomposition and knowledge transfer.