# Relighting Images in the Wild with a Self-Supervised Siamese Auto-Encoder

arXiv: https://arxiv.org/abs/2012.06444 · HTML: https://mlsanity.com/paper/2012.06444/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2020.

## Built on by (most cited first, 13 of 13)

- [NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images](https://mlsanity.com/paper/2305.17398/) (2023)
- [Neural Video Portrait Relighting in Real-time via Consistency Modeling](https://mlsanity.com/paper/2104.00484/) (2021)
- [Relighting4D: Neural Relightable Human from Videos](https://mlsanity.com/paper/2207.07104/) (2022)
- [Beyond Mono to Binaural: Generating Binaural Audio from Mono Audio with Depth and Cross Modal Attention](https://mlsanity.com/paper/2111.08046/) (2021)
- [Adversarial Relighting Against Face Recognition](https://mlsanity.com/paper/2108.07920/) (2021)
- [Comprehensive Relighting: Generalizable and Consistent Monocular Human Relighting and Harmonization](https://mlsanity.com/paper/2504.03011/) (2025)
- [SILT: Self-supervised Lighting Transfer Using Implicit Image Decomposition](https://mlsanity.com/paper/2110.12914/) (2021)
- [Rotation and Translation Invariant Representation Learning with Implicit Neural Representations](https://mlsanity.com/paper/2304.13995/) (2023)
- [NARRATE: A Normal Assisted Free-View Portrait Stylizer](https://mlsanity.com/paper/2207.00974/) (2022)
- [Neural Relighting and Expression Transfer On Video Portraits](https://mlsanity.com/paper/2107.14735/) (2021)
- [SIRfyN: Single Image Relighting from your Neighbors](https://mlsanity.com/paper/2112.04497/) (2021)
- [Deep Contrastive Patch-Based Subspace Learning for Camera Image Signal Processing](https://mlsanity.com/paper/2104.00253/) (2021)
- [ISG: I can See Your Gene Expression](https://mlsanity.com/paper/2210.16728/) (2022)

## Abstract (arXiv)

We propose a self-supervised method for image relighting of single view images in the wild. The method is based on an auto-encoder which deconstructs an image into two separate encodings, relating to the scene illumination and content, respectively. In order to disentangle this embedding information without supervision, we exploit the assumption that some augmentation operations do not affect the image content and only affect the direction of the light. A novel loss function, called spherical harmonic loss, is introduced that forces the illumination embedding to convert to a spherical harmonic vector. We train our model on large-scale datasets such as Youtube 8M and CelebA. Our experiments show that our method can correctly estimate scene illumination and realistically re-light input images, without any supervision or a prior shape model. Compared to supervised methods, our approach has similar performance and avoids common lighting artifacts.