# SIGNet: Intrinsic Image Decomposition by a Semantic and Invariant Gradient Driven Network for Indoor Scenes

arXiv: https://arxiv.org/abs/2208.14369 · HTML: https://mlsanity.com/paper/2208.14369/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 30 Aug 2022.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering](https://mlsanity.com/paper/1808.08601/) (2018)
- [OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets](https://mlsanity.com/paper/2007.12868/) (2020)
- [Neural Face Editing with Intrinsic Image Disentangling](https://mlsanity.com/paper/1704.04131/) (2017)
- [Learning single-image 3D reconstruction by generative modelling of shape, pose and shading](https://mlsanity.com/paper/1901.06447/) (2019)
- [Learning Non-Lambertian Object Intrinsics across ShapeNet Categories](https://mlsanity.com/paper/1612.08510/) (2016)
- [STAR: A Structure and Texture Aware Retinex Model](https://mlsanity.com/paper/1906.06690/) (2019)
- [Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression](https://mlsanity.com/paper/1512.02311/) (2015)
- [Joint Learning of Intrinsic Images and Semantic Segmentation](https://mlsanity.com/paper/1807.11857/) (2018)
- [Reflectance Adaptive Filtering Improves Intrinsic Image Estimation](https://mlsanity.com/paper/1612.05062/) (2016)
- [ShadingNet: Image Intrinsics by Fine-Grained Shading Decomposition](https://mlsanity.com/paper/1912.04023/) (2019)
- [Semantic Hierarchical Priors for Intrinsic Image Decomposition](https://mlsanity.com/paper/1902.03830/) (2019)

## Built on by (most cited first, 1 of 1)

- [JoIN: Joint GANs Inversion for Intrinsic Image Decomposition](https://mlsanity.com/paper/2305.11321/) (2023)

## Abstract (arXiv)

Intrinsic image decomposition (IID) is an under-constrained problem. Therefore, traditional approaches use hand crafted priors to constrain the problem. However, these constraints are limited when coping with complex scenes. Deep learning-based approaches learn these constraints implicitly through the data, but they often suffer from dataset biases (due to not being able to include all possible imaging conditions). In this paper, a combination of the two is proposed. Component specific priors like semantics and invariant features are exploited to obtain semantically and physically plausible reflectance transitions. These transitions are used to steer a progressive CNN with implicit homogeneity constraints to decompose reflectance and shading maps. An ablation study is conducted showing that the use of the proposed priors and progressive CNN increase the IID performance. State of the art performance on both our proposed dataset and the standard real-world IIW dataset shows the effectiveness of the proposed method. Code is made available at https://github.com/Morpheus3000/SIGNet