# Learning Restoration is Not Enough: Transfering Identical Mapping for Single-Image Shadow Removal

arXiv: https://arxiv.org/abs/2305.10640 · HTML: https://mlsanity.com/paper/2305.10640/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 28 of 28)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [Recurrent Feature Reasoning for Image Inpainting](https://mlsanity.com/paper/2008.03737/) (2020)
- [Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN](https://mlsanity.com/paper/1911.08718/) (2019)
- [Shadow Removal via Shadow Image Decomposition](https://mlsanity.com/paper/1908.08628/) (2019)
- [Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data](https://mlsanity.com/paper/1903.10683/) (2019)
- [RigNet: Repetitive Image Guided Network for Depth Completion](https://mlsanity.com/paper/2107.13802/) (2021)
- [ESIR: End-to-end Scene Text Recognition via Iterative Image Rectification](https://mlsanity.com/paper/1812.05824/) (2018)
- [Auto-Exposure Fusion for Single-Image Shadow Removal](https://mlsanity.com/paper/2103.01255/) (2021)
- [DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network](https://mlsanity.com/paper/2207.10434/) (2022)
- [From Shadow Segmentation to Shadow Removal](https://mlsanity.com/paper/2008.00267/) (2020)
- [Direction-aware Spatial Context Features for Shadow Detection and Removal](https://mlsanity.com/paper/1805.04635/) (2018)
- [From Shadow Generation to Shadow Removal](https://mlsanity.com/paper/2103.12997/) (2021)
- [Shadow Removal by a Lightness-Guided Network with Training on Unpaired Data](https://mlsanity.com/paper/2006.15617/) (2020)
- [RIS-GAN: Explore Residual and Illumination with Generative Adversarial Networks for Shadow Removal](https://mlsanity.com/paper/1911.09178/) (2019)
- [MISF: Multi-level Interactive Siamese Filtering for High-Fidelity Image Inpainting](https://mlsanity.com/paper/2203.06304/) (2022)
- [Cascaded Recurrent Neural Networks for Hyperspectral Image Classification](https://mlsanity.com/paper/1902.10858/) (2019)
- [Improving Shadow Suppression for Illumination Robust Face Recognition](https://mlsanity.com/paper/1710.05073/) (2017)
- [PAC-Bayes Information Bottleneck](https://mlsanity.com/paper/2109.14509/) (2021)
- [Learning from Synthetic Shadows for Shadow Detection and Removal](https://mlsanity.com/paper/2101.01713/) (2021)
- [Leveraging Inpainting for Single-Image Shadow Removal](https://mlsanity.com/paper/2302.05361/) (2023)
- [JPGNet: Joint Predictive Filtering and Generative Network for Image Inpainting](https://mlsanity.com/paper/2107.04281/) (2021)
- [Unsupervised Portrait Shadow Removal via Generative Priors](https://mlsanity.com/paper/2108.03466/) (2021)
- [The iterative convolution-thresholding method (ICTM) for image segmentation](https://mlsanity.com/paper/1904.10917/) (2019)
- [Benchmarking Shadow Removal for Facial Landmark Detection and Beyond](https://mlsanity.com/paper/2111.13790/) (2021)

## Built on by (most cited first, 2 of 2)

- [S3R-Net: A Single-Stage Approach to Self-Supervised Shadow Removal](https://mlsanity.com/paper/2404.12103/) (2024)
- [Leveraging Adaptive Implicit Representation Mapping for Ultra High-Resolution Image Segmentation](https://mlsanity.com/paper/2407.21256/) (2024)

## Abstract (arXiv)

Shadow removal is to restore shadow regions to their shadow-free counterparts while leaving non-shadow regions unchanged. State-of-the-art shadow removal methods train deep neural networks on collected shadow & shadow-free image pairs, which are desired to complete two distinct tasks via shared weights, i.e., data restoration for shadow regions and identical mapping for non-shadow regions. We find that these two tasks exhibit poor compatibility, and using shared weights for these two tasks could lead to the model being optimized towards only one task instead of both during the training process. Note that such a key issue is not identified by existing deep learning-based shadow removal methods. To address this problem, we propose to handle these two tasks separately and leverage the identical mapping results to guide the shadow restoration in an iterative manner. Specifically, our method consists of three components: an identical mapping branch (IMB) for non-shadow regions processing, an iterative de-shadow branch (IDB) for shadow regions restoration based on identical results, and a smart aggregation block (SAB). The IMB aims to reconstruct an image that is identical to the input one, which can benefit the restoration of the non-shadow regions without explicitly distinguishing between shadow and non-shadow regions. Utilizing the multi-scale features extracted by the IMB, the IDB can effectively transfer information from non-shadow regions to shadow regions progressively, facilitating the process of shadow removal. The SAB is designed to adaptive integrate features from both IMB and IDB. Moreover, it generates a finely tuned soft shadow mask that guides the process of removing shadows. Extensive experiments demonstrate our method outperforms all the state-of-the-art shadow removal approaches on the widely used shadow removal datasets.