# Auto-Exposure Fusion for Single-Image Shadow Removal

arXiv: https://arxiv.org/abs/2103.01255 · HTML: https://mlsanity.com/paper/2103.01255/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN](https://mlsanity.com/paper/1911.08718/) (2019)
- [Shadow Removal via Shadow Image Decomposition](https://mlsanity.com/paper/1908.08628/) (2019)
- [Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data](https://mlsanity.com/paper/1903.10683/) (2019)
- [DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs](https://mlsanity.com/paper/1712.07384/) (2017)
- [From Shadow Segmentation to Shadow Removal](https://mlsanity.com/paper/2008.00267/) (2020)
- [Direction-aware Spatial Context Features for Shadow Detection](https://mlsanity.com/paper/1712.04142/) (2017)
- [ARGAN: Attentive Recurrent Generative Adversarial Network for Shadow Detection and Removal](https://mlsanity.com/paper/1908.01323/) (2019)
- [EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining](https://mlsanity.com/paper/2009.09238/) (2020)
- [Improving Shadow Suppression for Illumination Robust Face Recognition](https://mlsanity.com/paper/1710.05073/) (2017)
- [Asymmetric GAN for Unpaired Image-to-image Translation](https://mlsanity.com/paper/1912.11660/) (2019)
- [Interactive Removal and Ground Truth for Difficult Shadow Scenes](https://mlsanity.com/paper/1608.00762/) (2016)

## Built on by (most cited first, 30 of 79)

- [ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal](https://mlsanity.com/paper/2212.04711/) (2022)
- [Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Models](https://mlsanity.com/paper/2304.08291/) (2023)
- [ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion](https://mlsanity.com/paper/2403.18818/) (2024)
- [Residual Denoising Diffusion Models](https://mlsanity.com/paper/2308.13712/) (2023)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [DeS3: Adaptive Attention-driven Self and Soft Shadow Removal using ViT Similarity](https://mlsanity.com/paper/2211.08089/) (2022)
- [High-Resolution Document Shadow Removal via A Large-Scale Real-World Dataset and A Frequency-Aware Shadow Erasing Net](https://mlsanity.com/paper/2308.14221/) (2023)
- [MISF: Multi-level Interactive Siamese Filtering for High-Fidelity Image Inpainting](https://mlsanity.com/paper/2203.06304/) (2022)
- [ShadowFormer: Global Context Helps Image Shadow Removal](https://mlsanity.com/paper/2302.01650/) (2023)
- [Latent Feature-Guided Diffusion Models for Shadow Removal](https://mlsanity.com/paper/2312.02156/) (2023)
- [Leveraging Inpainting for Single-Image Shadow Removal](https://mlsanity.com/paper/2302.05361/) (2023)
- [Recasting Regional Lighting for Shadow Removal](https://mlsanity.com/paper/2402.00341/) (2024)
- [Instance Shadow Detection with A Single-Stage Detector](https://mlsanity.com/paper/2207.04614/) (2022)
- [ShadowRefiner: Towards Mask-free Shadow Removal via Fast Fourier Transformer](https://mlsanity.com/paper/2406.02559/) (2024)
- [Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise](https://mlsanity.com/paper/2311.14900/) (2023)
- [OmniSR: Shadow Removal under Direct and Indirect Lighting](https://mlsanity.com/paper/2410.01719/) (2024)
- [Blind Image Decomposition](https://mlsanity.com/paper/2108.11364/) (2021)
- [JPGNet: Joint Predictive Filtering and Generative Network for Image Inpainting](https://mlsanity.com/paper/2107.04281/) (2021)
- [ShaDocNet: Learning Spatial-Aware Tokens in Transformer for Document Shadow Removal](https://mlsanity.com/paper/2211.16675/) (2022)
- [Detail-Preserving Latent Diffusion for Stable Shadow Removal](https://mlsanity.com/paper/2412.17630/) (2024)
- [Regional Attention for Shadow Removal](https://mlsanity.com/paper/2411.14201/) (2024)
- [Let There be Light: Improved Traffic Surveillance via Detail Preserving Night-to-Day Transfer](https://mlsanity.com/paper/2105.05011/) (2021)
- [Shadow-Aware Dynamic Convolution for Shadow Removal](https://mlsanity.com/paper/2205.04908/) (2022)
- [Diff-Shadow: Global-guided Diffusion Model for Shadow Removal](https://mlsanity.com/paper/2407.16214/) (2024)
- [DocDeshadower: Frequency-Aware Transformer for Document Shadow Removal](https://mlsanity.com/paper/2307.15318/) (2023)
- [Towards Image Ambient Lighting Normalization](https://mlsanity.com/paper/2403.18730/) (2024)
- [CRFormer: A Cross-Region Transformer for Shadow Removal](https://mlsanity.com/paper/2207.01600/) (2022)
- [Memory augment is All You Need for image restoration](https://mlsanity.com/paper/2309.01377/) (2023)
- [SpA-Former: Transformer image shadow detection and removal via spatial attention](https://mlsanity.com/paper/2206.10910/) (2022)
- [ShadowMaskFormer: Mask Augmented Patch Embeddings for Shadow Removal](https://mlsanity.com/paper/2404.18433/) (2024)

## Abstract (arXiv)

Shadow removal is still a challenging task due to its inherent background-dependent and spatial-variant properties, leading to unknown and diverse shadow patterns. Even powerful state-of-the-art deep neural networks could hardly recover traceless shadow-removed background. This paper proposes a new solution for this task by formulating it as an exposure fusion problem to address the challenges. Intuitively, we can first estimate multiple over-exposure images w.r.t. the input image to let the shadow regions in these images have the same color with shadow-free areas in the input image. Then, we fuse the original input with the over-exposure images to generate the final shadow-free counterpart. Nevertheless, the spatial-variant property of the shadow requires the fusion to be sufficiently `smart', that is, it should automatically select proper over-exposure pixels from different images to make the final output natural. To address this challenge, we propose the shadow-aware FusionNet that takes the shadow image as input to generate fusion weight maps across all the over-exposure images. Moreover, we propose the boundary-aware RefineNet to eliminate the remaining shadow trace further. We conduct extensive experiments on the ISTD, ISTD+, and SRD datasets to validate our method's effectiveness and show better performance in shadow regions and comparable performance in non-shadow regions over the state-of-the-art methods. We release the model and code in https://github.com/tsingqguo/exposure-fusion-shadow-removal.