# Leveraging Inpainting for Single-Image Shadow Removal

arXiv: https://arxiv.org/abs/2302.05361 · HTML: https://mlsanity.com/paper/2302.05361/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 10 Feb 2023.

## Builds on (most cited first, 27 of 27)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN](https://mlsanity.com/paper/1911.08718/) (2019)
- [Shadow Removal via Shadow Image Decomposition](https://mlsanity.com/paper/1908.08628/) (2019)
- [Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data](https://mlsanity.com/paper/1903.10683/) (2019)
- [Auto-Exposure Fusion for Single-Image Shadow Removal](https://mlsanity.com/paper/2103.01255/) (2021)
- [DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network](https://mlsanity.com/paper/2207.10434/) (2022)
- [From Shadow Segmentation to Shadow Removal](https://mlsanity.com/paper/2008.00267/) (2020)
- [Direction-aware Spatial Context Features for Shadow Detection and Removal](https://mlsanity.com/paper/1805.04635/) (2018)
- [From Shadow Generation to Shadow Removal](https://mlsanity.com/paper/2103.12997/) (2021)
- [CANet: A Context-Aware Network for Shadow Removal](https://mlsanity.com/paper/2108.09894/) (2021)
- [EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining](https://mlsanity.com/paper/2009.09238/) (2020)
- [Physics-based Shadow Image Decomposition for Shadow Removal](https://mlsanity.com/paper/2012.13018/) (2020)
- [Shadow Removal by a Lightness-Guided Network with Training on Unpaired Data](https://mlsanity.com/paper/2006.15617/) (2020)
- [MISF: Multi-level Interactive Siamese Filtering for High-Fidelity Image Inpainting](https://mlsanity.com/paper/2203.06304/) (2022)
- [Improving Shadow Suppression for Illumination Robust Face Recognition](https://mlsanity.com/paper/1710.05073/) (2017)
- [PAC-Bayes Information Bottleneck](https://mlsanity.com/paper/2109.14509/) (2021)
- [Learning from Synthetic Shadows for Shadow Detection and Removal](https://mlsanity.com/paper/2101.01713/) (2021)
- [JPGNet: Joint Predictive Filtering and Generative Network for Image Inpainting](https://mlsanity.com/paper/2107.04281/) (2021)
- [Interactive Removal and Ground Truth for Difficult Shadow Scenes](https://mlsanity.com/paper/1608.00762/) (2016)
- [Appearance Harmonization for Single Image Shadow Removal](https://mlsanity.com/paper/1603.06398/) (2016)
- [Benchmarking Shadow Removal for Facial Landmark Detection and Beyond](https://mlsanity.com/paper/2111.13790/) (2021)

## Built on by (most cited first, 25 of 25)

- [OmniSR: Shadow Removal under Direct and Indirect Lighting](https://mlsanity.com/paper/2410.01719/) (2024)
- [Regional Attention for Shadow Removal](https://mlsanity.com/paper/2411.14201/) (2024)
- [ShadowMaskFormer: Mask Augmented Patch Embeddings for Shadow Removal](https://mlsanity.com/paper/2404.18433/) (2024)
- [Single-Image Shadow Removal Using Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2407.08865/) (2024)
- [Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Era](https://mlsanity.com/paper/2409.02108/) (2024)
- [S3R-Net: A Single-Stage Approach to Self-Supervised Shadow Removal](https://mlsanity.com/paper/2404.12103/) (2024)
- [Shadow Removal Refinement via Material-Consistent Shadow Edges](https://mlsanity.com/paper/2409.06848/) (2024)
- [SoftShadow: Leveraging Soft Masks for Penumbra-Aware Shadow Removal](https://mlsanity.com/paper/2409.07041/) (2024)
- [ShadowHack: Hacking Shadows via Luminance-Color Divide and Conquer](https://mlsanity.com/paper/2412.02545/) (2024)
- [Learning Restoration is Not Enough: Transfering Identical Mapping for Single-Image Shadow Removal](https://mlsanity.com/paper/2305.10640/) (2023)
- [Prompt-Aware Controllable Shadow Removal](https://mlsanity.com/paper/2501.15043/) (2025)
- [Leveraging Adaptive Implicit Representation Mapping for Ultra High-Resolution Image Segmentation](https://mlsanity.com/paper/2407.21256/) (2024)
- [ShadowMamba: State-Space Model with Boundary-Region Selective Scan for Shadow Removal](https://mlsanity.com/paper/2411.03260/) (2024)
- [Controlling the Latent Diffusion Model for Generative Image Shadow Removal via Residual Generation](https://mlsanity.com/paper/2412.02322/) (2024)
- [FASR-Net: Unsupervised Shadow Removal Leveraging Inherent Frequency Priors](https://mlsanity.com/paper/2504.05779/) (2025)
- [Structure-Guided Diffusion Models for High-Fidelity Portrait Shadow Removal](https://mlsanity.com/paper/2507.04692/) (2025)
- [DeshadowMamba: Deshadowing as 1D Sequential Similarity](https://mlsanity.com/paper/2510.24260/) (2025)
- [Domain-Grounded Candidate Selection for Agentic Image Editing: A Shadow Removal Case](https://mlsanity.com/paper/2608.06075/) (2026)
- [CLII: Visual-Text Inpainting via Cross-Modal Predictive Interaction](https://mlsanity.com/paper/2407.16204/) (2024)
- [Image Restoration via Multi-domain Learning](https://mlsanity.com/paper/2505.05504/) (2025)
- [Contrast-Prior Enhanced Duality for Mask-Free Shadow Removal](https://mlsanity.com/paper/2507.21949/) (2025)
- [D2-Mamba: Dual-Scale Fusion and Dual-Path Scanning with SSMs for Shadow Removal](https://mlsanity.com/paper/2508.12750/) (2025)
- [Cast and Attached Shadow Detection via Iterative Light and Geometry Reasoning](https://mlsanity.com/paper/2512.06179/) (2025)
- [After a Decade: Bringing Shadow Removal into the Real World with Agentic Training Data](https://mlsanity.com/paper/2609.38607/) (2026)
- [Shadow Feature Refinement Network: Progressive Feature Refinement based on Knowledge Distillation for Effective Shadow Removal](https://mlsanity.com/paper/2610.05325/) (2026)

## Abstract (arXiv)

Fully-supervised shadow removal methods achieve the best restoration qualities on public datasets but still generate some shadow remnants. One of the reasons is the lack of large-scale shadow & shadow-free image pairs. Unsupervised methods can alleviate the issue but their restoration qualities are much lower than those of fully-supervised methods. In this work, we find that pretraining shadow removal networks on the image inpainting dataset can reduce the shadow remnants significantly: a naive encoder-decoder network gets competitive restoration quality w.r.t. the state-of-the-art methods via only 10% shadow & shadow-free image pairs. After analyzing networks with/without inpainting pre-training via the information stored in the weight (IIW), we find that inpainting pretraining improves restoration quality in non-shadow regions and enhances the generalization ability of networks significantly. Additionally, shadow removal fine-tuning enables networks to fill in the details of shadow regions. Inspired by these observations we formulate shadow removal as an adaptive fusion task that takes advantage of both shadow removal and image inpainting. Specifically, we develop an adaptive fusion network consisting of two encoders, an adaptive fusion block, and a decoder. The two encoders are responsible for extracting the feature from the shadow image and the shadow-masked image respectively. The adaptive fusion block is responsible for combining these features in an adaptive manner. Finally, the decoder converts the adaptive fused features to the desired shadow-free result. The extensive experiments show that our method empowered with inpainting outperforms all state-of-the-art methods.