# PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decomposition

arXiv: https://arxiv.org/abs/2203.16670 · HTML: https://mlsanity.com/paper/2203.16670/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 61 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 23 of 23)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [InverseRenderNet: Learning single image inverse rendering](https://mlsanity.com/paper/1811.12328/) (2018)
- [CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering](https://mlsanity.com/paper/1808.08601/) (2018)
- [Learning Intrinsic Image Decomposition from Watching the World](https://mlsanity.com/paper/1804.00582/) (2018)
- [Neural Face Editing with Intrinsic Image Disentangling](https://mlsanity.com/paper/1704.04131/) (2017)
- [Unsupervised Learning for Intrinsic Image Decomposition from a Single Image](https://mlsanity.com/paper/1911.09930/) (2019)
- [Self-Supervised Intrinsic Image Decomposition](https://mlsanity.com/paper/1711.03678/) (2017)
- [Learning single-image 3D reconstruction by generative modelling of shape, pose and shading](https://mlsanity.com/paper/1901.06447/) (2019)
- [Learning Non-Lambertian Object Intrinsics across ShapeNet Categories](https://mlsanity.com/paper/1612.08510/) (2016)
- [STAR: A Structure and Texture Aware Retinex Model](https://mlsanity.com/paper/1906.06690/) (2019)
- [Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression](https://mlsanity.com/paper/1512.02311/) (2015)
- [Joint Learning of Intrinsic Images and Semantic Segmentation](https://mlsanity.com/paper/1807.11857/) (2018)
- [Learning Data-driven Reflectance Priors for Intrinsic Image Decomposition](https://mlsanity.com/paper/1510.02413/) (2015)
- [Unsupervised Deep Single-Image Intrinsic Decomposition using Illumination-Varying Image Sequences](https://mlsanity.com/paper/1803.00805/) (2018)
- [CNN based Learning using Reflection and Retinex Models for Intrinsic Image Decomposition](https://mlsanity.com/paper/1712.01056/) (2017)
- [Reflectance Adaptive Filtering Improves Intrinsic Image Estimation](https://mlsanity.com/paper/1612.05062/) (2016)
- [ShadingNet: Image Intrinsics by Fine-Grained Shading Decomposition](https://mlsanity.com/paper/1912.04023/) (2019)
- [Intrinsic Image Transformation via Scale Space Decomposition](https://mlsanity.com/paper/1805.10253/) (2018)
- [Physics-based Shading Reconstruction for Intrinsic Image Decomposition](https://mlsanity.com/paper/2009.01540/) (2020)

## Built on by (most cited first, 26 of 26)

- [Intrinsic Image Decomposition via Ordinal Shading](https://mlsanity.com/paper/2311.12792/) (2023)
- [Colorful Diffuse Intrinsic Image Decomposition in the Wild](https://mlsanity.com/paper/2409.13690/) (2024)
- [Exploiting Diffusion Prior for Generalizable Dense Prediction](https://mlsanity.com/paper/2311.18832/) (2023)
- [LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting](https://mlsanity.com/paper/2412.00177/) (2024)
- [ScribbleLight: Single Image Indoor Relighting with Scribbles](https://mlsanity.com/paper/2411.17696/) (2024)
- [Estimating Reflectance Layer from A Single Image: Integrating Reflectance Guidance and Shadow/Specular Aware Learning](https://mlsanity.com/paper/2211.14751/) (2022)
- [DNF-Intrinsic: Deterministic Noise-Free Diffusion for Indoor Inverse Rendering](https://mlsanity.com/paper/2507.03924/) (2025)
- [You Only Look Around: Learning Illumination Invariant Feature for Low-light Object Detection](https://mlsanity.com/paper/2410.18398/) (2024)
- [Few-shot Neural Radiance Fields Under Unconstrained Illumination](https://mlsanity.com/paper/2303.11728/) (2023)
- [Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theory](https://mlsanity.com/paper/2407.20785/) (2024)
- [Joint RGB-Spectral Decomposition Model Guided Image Enhancement in Mobile Photography](https://mlsanity.com/paper/2407.17996/) (2024)
- [JoIN: Joint GANs Inversion for Intrinsic Image Decomposition](https://mlsanity.com/paper/2305.11321/) (2023)
- [LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction under Multi-illumination Conditions](https://mlsanity.com/paper/2602.01118/) (2026)
- [DPF: Learning Dense Prediction Fields with Weak Supervision](https://mlsanity.com/paper/2303.16890/) (2023)
- [A General Albedo Recovery Approach for Aerial Photogrammetric Images through Inverse Rendering](https://mlsanity.com/paper/2409.03032/) (2024)
- [NeRF synthesis with shading guidance](https://mlsanity.com/paper/2306.11556/) (2023)
- [MLI-NeRF: Multi-Light Intrinsic-Aware Neural Radiance Fields](https://mlsanity.com/paper/2411.17235/) (2024)
- [FlowIID: Single-Step Intrinsic Image Decomposition via Latent Flow Matching](https://mlsanity.com/paper/2601.12329/) (2026)
- [Shape from Shading for Robotic Manipulation](https://mlsanity.com/paper/2304.11824/) (2023)
- [Intrinsic Image Decomposition Using Point Cloud Representation](https://mlsanity.com/paper/2307.10924/) (2023)
- [Learning Relighting and Intrinsic Decomposition in Neural Radiance Fields](https://mlsanity.com/paper/2406.11077/) (2024)
- [Incorporating dense metric depth into neural 3D representations for view synthesis and relighting](https://mlsanity.com/paper/2409.03061/) (2024)
- [Scalable and Realistic Virtual Try-on Application for Foundation Makeup with Kubelka-Munk Theory](https://mlsanity.com/paper/2507.07333/) (2025)
- [Clair Obscur: an Illumination-Aware Method for Real-World Image Vectorization](https://mlsanity.com/paper/2511.20034/) (2025)
- [Relighting as a Probe of Visual Priors via Augmented Latent Intrinsics](https://mlsanity.com/paper/2602.01391/) (2026)
- [Albedo Estimation via Latent Bridge Matching](https://mlsanity.com/paper/2609.09884/) (2026)

## Abstract (arXiv)

Intrinsic image decomposition is the process of recovering the image formation components (reflectance and shading) from an image. Previous methods employ either explicit priors to constrain the problem or implicit constraints as formulated by their losses (deep learning). These methods can be negatively influenced by strong illumination conditions causing shading-reflectance leakages. Therefore, in this paper, an end-to-end edge-driven hybrid CNN approach is proposed for intrinsic image decomposition. Edges correspond to illumination invariant gradients. To handle hard negative illumination transitions, a hierarchical approach is taken including global and local refinement layers. We make use of attention layers to further strengthen the learning process. An extensive ablation study and large scale experiments are conducted showing that it is beneficial for edge-driven hybrid IID networks to make use of illumination invariant descriptors and that separating global and local cues helps in improving the performance of the network. Finally, it is shown that the proposed method obtains state of the art performance and is able to generalise well to real world images. The project page with pretrained models, finetuned models and network code can be found at https://ivi.fnwi.uva.nl/cv/pienet/.