# SILT: Self-supervised Lighting Transfer Using Implicit Image Decomposition

arXiv: https://arxiv.org/abs/2110.12914 · HTML: https://mlsanity.com/paper/2110.12914/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 30 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 25 of 25)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [InverseRenderNet: Learning single image inverse rendering](https://mlsanity.com/paper/1811.12328/) (2018)
- [CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering](https://mlsanity.com/paper/1808.08601/) (2018)
- [Learning Intrinsic Image Decomposition from Watching the World](https://mlsanity.com/paper/1804.00582/) (2018)
- [Unsupervised Learning for Intrinsic Image Decomposition from a Single Image](https://mlsanity.com/paper/1911.09930/) (2019)
- [Self-supervised Outdoor Scene Relighting](https://mlsanity.com/paper/2107.03106/) (2021)
- [Learning to Factorize and Relight a City](https://mlsanity.com/paper/2008.02796/) (2020)
- [VIDIT: Virtual Image Dataset for Illumination Transfer](https://mlsanity.com/paper/2005.05460/) (2020)
- [Learning Data-driven Reflectance Priors for Intrinsic Image Decomposition](https://mlsanity.com/paper/1510.02413/) (2015)
- [NTIRE 2021 Depth Guided Image Relighting Challenge](https://mlsanity.com/paper/2104.13365/) (2021)
- [Reflectance Adaptive Filtering Improves Intrinsic Image Estimation](https://mlsanity.com/paper/1612.05062/) (2016)
- [AIM 2020: Scene Relighting and Illumination Estimation Challenge](https://mlsanity.com/paper/2009.12798/) (2020)
- [WDRN : A Wavelet Decomposed RelightNet for Image Relighting](https://mlsanity.com/paper/2009.06678/) (2020)
- [Relighting Images in the Wild with a Self-Supervised Siamese Auto-Encoder](https://mlsanity.com/paper/2012.06444/) (2020)
- [Deep Relighting Networks for Image Light Source Manipulation](https://mlsanity.com/paper/2008.08298/) (2020)
- [2D Image Relighting with Image-to-Image Translation](https://mlsanity.com/paper/2006.07816/) (2020)
- [Physics-based Shading Reconstruction for Intrinsic Image Decomposition](https://mlsanity.com/paper/2009.01540/) (2020)
- [DSRN: an Efficient Deep Network for Image Relighting](https://mlsanity.com/paper/2102.09242/) (2021)
- [MSR-Net: Multi-Scale Relighting Network for One-to-One Relighting](https://mlsanity.com/paper/2107.06125/) (2021)

## Built on by (most cited first, 4 of 4)

- [S3R-Net: A Single-Stage Approach to Self-Supervised Shadow Removal](https://mlsanity.com/paper/2404.12103/) (2024)
- [Relighting from a Single Image: Datasets and Deep Intrinsic-based Architecture](https://mlsanity.com/paper/2409.18770/) (2024)
- [SIRfyN: Single Image Relighting from your Neighbors](https://mlsanity.com/paper/2112.04497/) (2021)
- [ISG: I can See Your Gene Expression](https://mlsanity.com/paper/2210.16728/) (2022)

## Abstract (arXiv)

We present SILT, a Self-supervised Implicit Lighting Transfer method. Unlike previous research on scene relighting, we do not seek to apply arbitrary new lighting configurations to a given scene. Instead, we wish to transfer the lighting style from a database of other scenes, to provide a uniform lighting style regardless of the input. The solution operates as a two-branch network that first aims to map input images of any arbitrary lighting style to a unified domain, with extra guidance achieved through implicit image decomposition. We then remap this unified input domain using a discriminator that is presented with the generated outputs and the style reference, i.e. images of the desired illumination conditions. Our method is shown to outperform supervised relighting solutions across two different datasets without requiring lighting supervision.