# DeRenderNet: Intrinsic Image Decomposition of Urban Scenes with Shape-(In)dependent Shading Rendering

arXiv: https://arxiv.org/abs/2104.13602 · HTML: https://mlsanity.com/paper/2104.13602/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 28 Apr 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Improving Semantic Segmentation via Video Propagation and Label Relaxation](https://mlsanity.com/paper/1812.01593/) (2018)
- [SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild](https://mlsanity.com/paper/1712.01261/) (2017)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [InverseRenderNet: Learning single image inverse rendering](https://mlsanity.com/paper/1811.12328/) (2018)
- [CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering](https://mlsanity.com/paper/1808.08601/) (2018)
- [Learning Intrinsic Image Decomposition from Watching the World](https://mlsanity.com/paper/1804.00582/) (2018)
- [Deep Sky Modeling for Single Image Outdoor Lighting Estimation](https://mlsanity.com/paper/1905.03897/) (2019)
- [Deep Shading: Convolutional Neural Networks for Screen-Space Shading](https://mlsanity.com/paper/1603.06078/) (2016)
- [Learning Non-Lambertian Object Intrinsics across ShapeNet Categories](https://mlsanity.com/paper/1612.08510/) (2016)
- [Learning to Factorize and Relight a City](https://mlsanity.com/paper/2008.02796/) (2020)
- [Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks](https://mlsanity.com/paper/1612.07429/) (2016)
- [Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression](https://mlsanity.com/paper/1512.02311/) (2015)
- [LIME: Live Intrinsic Material Estimation](https://mlsanity.com/paper/1801.01075/) (2018)
- [Learning Data-driven Reflectance Priors for Intrinsic Image Decomposition](https://mlsanity.com/paper/1510.02413/) (2015)
- [Shading Annotations in the Wild](https://mlsanity.com/paper/1705.01156/) (2017)
- [Reflectance Adaptive Filtering Improves Intrinsic Image Estimation](https://mlsanity.com/paper/1612.05062/) (2016)
- [Learning to Estimate Indoor Lighting from 3D Objects](https://mlsanity.com/paper/1806.03994/) (2018)

## Built on by (most cited first, 6 of 6)

- [Physically-Based Editing of Indoor Scene Lighting from a Single Image](https://mlsanity.com/paper/2205.09343/) (2022)
- [Unsupervised Intrinsic Image Decomposition with LiDAR Intensity](https://mlsanity.com/paper/2303.10820/) (2023)
- [LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction under Multi-illumination Conditions](https://mlsanity.com/paper/2602.01118/) (2026)
- [Discriminative feature encoding for intrinsic image decomposition](https://mlsanity.com/paper/2209.12155/) (2022)
- [Unsupervised Intrinsic Image Decomposition with LiDAR Intensity Enhanced Training](https://mlsanity.com/paper/2403.14089/) (2024)
- [BRDFusion: Physics Meets Generation for Urban Scene Inverse Rendering](https://mlsanity.com/paper/2606.17049/) (2026)

## Abstract (arXiv)

We propose DeRenderNet, a deep neural network to decompose the albedo and latent lighting, and render shape-(in)dependent shadings, given a single image of an outdoor urban scene, trained in a self-supervised manner. To achieve this goal, we propose to use the albedo maps extracted from scenes in videogames as direct supervision and pre-compute the normal and shadow prior maps based on the depth maps provided as indirect supervision. Compared with state-of-the-art intrinsic image decomposition methods, DeRenderNet produces shadow-free albedo maps with clean details and an accurate prediction of shadows in the shape-independent shading, which is shown to be effective in re-rendering and improving the accuracy of high-level vision tasks for urban scenes.