# Multi-modal Bifurcated Network for Depth Guided Image Relighting

arXiv: https://arxiv.org/abs/2105.00690 · HTML: https://mlsanity.com/paper/2105.00690/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 3 May 2021.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06227/) (2020)
- [A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06811/) (2020)
- [Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2008.07064/) (2020)
- [RGB-D Salient Object Detection via 3D Convolutional Neural Networks](https://mlsanity.com/paper/2101.10241/) (2021)
- [VIDIT: Virtual Image Dataset for Illumination Transfer](https://mlsanity.com/paper/2005.05460/) (2020)
- [NTIRE 2021 Depth Guided Image Relighting Challenge](https://mlsanity.com/paper/2104.13365/) (2021)
- [AIM 2020: Scene Relighting and Illumination Estimation Challenge](https://mlsanity.com/paper/2009.12798/) (2020)
- [Y-net: Multi-scale feature aggregation network with wavelet structure similarity loss function for single image dehazing](https://mlsanity.com/paper/2003.13912/) (2020)
- [WDRN : A Wavelet Decomposed RelightNet for Image Relighting](https://mlsanity.com/paper/2009.06678/) (2020)
- [Wavelet Channel Attention Module with a Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2007.09163/) (2020)

## Built on by (most cited first, 6 of 6)

- [LAFFNet: A Lightweight Adaptive Feature Fusion Network for Underwater Image Enhancement](https://mlsanity.com/paper/2105.01299/) (2021)
- [Designing An Illumination-Aware Network for Deep Image Relighting](https://mlsanity.com/paper/2207.10582/) (2022)
- [Local Relighting of Real Scenes](https://mlsanity.com/paper/2207.02774/) (2022)
- [Robust Unsupervised Multi-Object Tracking in Noisy Environments](https://mlsanity.com/paper/2105.10005/) (2021)
- [SIRfyN: Single Image Relighting from your Neighbors](https://mlsanity.com/paper/2112.04497/) (2021)
- [Zero-Shot Depth from Defocus](https://mlsanity.com/paper/2603.26658/) (2026)

## Abstract (arXiv)

Image relighting aims to recalibrate the illumination setting in an image. In this paper, we propose a deep learning-based method called multi-modal bifurcated network (MBNet) for depth guided image relighting. That is, given an image and the corresponding depth maps, a new image with the given illuminant angle and color temperature is generated by our network. This model extracts the image and the depth features by the bifurcated network in the encoder. To use the two features effectively, we adopt the dynamic dilated pyramid modules in the decoder. Moreover, to increase the variety of training data, we propose a novel data process pipeline to increase the number of the training data. Experiments conducted on the VIDIT dataset show that the proposed solution obtains the 1st place in terms of SSIM and PMS in the NTIRE 2021 Depth Guide One-to-one Relighting Challenge.