# CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering

arXiv: https://arxiv.org/abs/1808.08601 · HTML: https://mlsanity.com/paper/1808.08601/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2018.

## Built on by (most cited first, 30 of 85)

- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenes](https://mlsanity.com/paper/2304.03266/) (2023)
- [Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting](https://mlsanity.com/paper/2109.06061/) (2021)
- [Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing](https://mlsanity.com/paper/2211.03017/) (2022)
- [Intrinsic Image Diffusion for Indoor Single-view Material Estimation](https://mlsanity.com/paper/2312.12274/) (2023)
- [IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination](https://mlsanity.com/paper/2404.11593/) (2024)
- [Physically-Based Editing of Indoor Scene Lighting from a Single Image](https://mlsanity.com/paper/2205.09343/) (2022)
- [IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes](https://mlsanity.com/paper/2206.08423/) (2022)
- [DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models](https://mlsanity.com/paper/2501.18590/) (2025)
- [Intrinsic Image Decomposition via Ordinal Shading](https://mlsanity.com/paper/2311.12792/) (2023)
- [IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis](https://mlsanity.com/paper/2210.00647/) (2022)
- [Colorful Diffuse Intrinsic Image Decomposition in the Wild](https://mlsanity.com/paper/2409.13690/) (2024)
- [UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting](https://mlsanity.com/paper/2506.15673/) (2025)
- [A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyond](https://mlsanity.com/paper/2112.03842/) (2021)
- [Material Palette: Extraction of Materials from a Single Image](https://mlsanity.com/paper/2311.17060/) (2023)
- [Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering](https://mlsanity.com/paper/2408.09702/) (2024)
- [PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decomposition](https://mlsanity.com/paper/2203.16670/) (2022)
- [Multi-view Inverse Rendering for Large-scale Real-world Indoor Scenes](https://mlsanity.com/paper/2211.10206/) (2022)
- [ScribbleLight: Single Image Indoor Relighting with Scribbles](https://mlsanity.com/paper/2411.17696/) (2024)
- [IRIS: Inverse Rendering of Indoor Scenes from Low Dynamic Range Images](https://mlsanity.com/paper/2401.12977/) (2024)
- [Outdoor inverse rendering from a single image using multiview self-supervision](https://mlsanity.com/paper/2102.06591/) (2021)
- [Estimating Reflectance Layer from A Single Image: Integrating Reflectance Guidance and Shadow/Specular Aware Learning](https://mlsanity.com/paper/2211.14751/) (2022)
- [Latent Intrinsics Emerge from Training to Relight](https://mlsanity.com/paper/2405.21074/) (2024)
- [Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation](https://mlsanity.com/paper/2207.06899/) (2022)
- [Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluation](https://mlsanity.com/paper/2306.15662/) (2023)
- [LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fields](https://mlsanity.com/paper/2504.20026/) (2025)
- [Weakly-supervised Single-view Image Relighting](https://mlsanity.com/paper/2303.13852/) (2023)
- [MAIR: Multi-view Attention Inverse Rendering with 3D Spatially-Varying Lighting Estimation](https://mlsanity.com/paper/2303.12368/) (2023)
- [Lighting, Reflectance and Geometry Estimation from 360circ Panoramic Stereo](https://mlsanity.com/paper/2104.09886/) (2021)
- [Concept Distillation: Leveraging Human-Centered Explanations for Model Improvement](https://mlsanity.com/paper/2311.15303/) (2023)

## Abstract (arXiv)

Intrinsic image decomposition is a challenging, long-standing computer vision problem for which ground truth data is very difficult to acquire. We explore the use of synthetic data for training CNN-based intrinsic image decomposition models, then applying these learned models to real-world images. To that end, we present ICG, a new, large-scale dataset of physically-based rendered images of scenes with full ground truth decompositions. The rendering process we use is carefully designed to yield high-quality, realistic images, which we find to be crucial for this problem domain. We also propose a new end-to-end training method that learns better decompositions by leveraging ICG, and optionally IIW and SAW, two recent datasets of sparse annotations on real-world images. Surprisingly, we find that a decomposition network trained solely on our synthetic data outperforms the state-of-the-art on both IIW and SAW, and performance improves even further when IIW and SAW data is added during training. Our work demonstrates the suprising effectiveness of carefully-rendered synthetic data for the intrinsic images task.