# S3Net: A Single Stream Structure for Depth Guided Image Relighting

arXiv: https://arxiv.org/abs/2105.00681 · HTML: https://mlsanity.com/paper/2105.00681/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 3 May 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Recurrent Models of Visual Attention](https://mlsanity.com/paper/1406.6247/) (2014)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation](https://mlsanity.com/paper/2007.09183/) (2020)
- [Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06227/) (2020)
- [Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2008.07064/) (2020)
- [VIDIT: Virtual Image Dataset for Illumination Transfer](https://mlsanity.com/paper/2005.05460/) (2020)
- [NTIRE 2021 Depth Guided Image Relighting Challenge](https://mlsanity.com/paper/2104.13365/) (2021)
- [AIM 2020: Scene Relighting and Illumination Estimation Challenge](https://mlsanity.com/paper/2009.12798/) (2020)
- [Y-net: Multi-scale feature aggregation network with wavelet structure similarity loss function for single image dehazing](https://mlsanity.com/paper/2003.13912/) (2020)
- [WDRN : A Wavelet Decomposed RelightNet for Image Relighting](https://mlsanity.com/paper/2009.06678/) (2020)
- [Neural Stereoscopic Image Style Transfer](https://mlsanity.com/paper/1802.09985/) (2018)
- [Wavelet Channel Attention Module with a Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2007.09163/) (2020)

## Built on by (most cited first, 9 of 9)

- [LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting](https://mlsanity.com/paper/2412.00177/) (2024)
- [GenLit: Reformulating Single-Image Relighting as Video Generation](https://mlsanity.com/paper/2412.11224/) (2024)
- [Latent Intrinsics Emerge from Training to Relight](https://mlsanity.com/paper/2405.21074/) (2024)
- [LAFFNet: A Lightweight Adaptive Feature Fusion Network for Underwater Image Enhancement](https://mlsanity.com/paper/2105.01299/) (2021)
- [Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography](https://mlsanity.com/paper/2212.12324/) (2022)
- [Robust Unsupervised Multi-Object Tracking in Noisy Environments](https://mlsanity.com/paper/2105.10005/) (2021)
- [Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation](https://mlsanity.com/paper/2609.08084/) (2026)
- [CLIP-RL: Surgical Scene Segmentation Using Contrastive Language-Vision Pretraining & Reinforcement Learning](https://mlsanity.com/paper/2507.04317/) (2025)
- [Relighting as a Probe of Visual Priors via Augmented Latent Intrinsics](https://mlsanity.com/paper/2602.01391/) (2026)

## Abstract (arXiv)

Depth guided any-to-any image relighting aims to generate a relit image from the original image and corresponding depth maps to match the illumination setting of the given guided image and its depth map. To the best of our knowledge, this task is a new challenge that has not been addressed in the previous literature. To address this issue, we propose a deep learning-based neural Single Stream Structure network called S3Net for depth guided image relighting. This network is an encoder-decoder model. We concatenate all images and corresponding depth maps as the input and feed them into the model. The decoder part contains the attention module and the enhanced module to focus on the relighting-related regions in the guided images. Experiments performed on challenging benchmark show that the proposed model achieves the 3 rd highest SSIM in the NTIRE 2021 Depth Guided Any-to-any Relighting Challenge.