# Physics-based Shadow Image Decomposition for Shadow Removal

arXiv: https://arxiv.org/abs/2012.13018 · HTML: https://mlsanity.com/paper/2012.13018/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2020.

## Built on by (most cited first, 30 of 39)

- [Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon](https://mlsanity.com/paper/2203.03818/) (2022)
- [Zero-shot Object Counting](https://mlsanity.com/paper/2303.02001/) (2023)
- [Generating Representative Samples for Few-Shot Classification](https://mlsanity.com/paper/2205.02918/) (2022)
- [DeS3: Adaptive Attention-driven Self and Soft Shadow Removal using ViT Similarity](https://mlsanity.com/paper/2211.08089/) (2022)
- [High-Resolution Document Shadow Removal via A Large-Scale Real-World Dataset and A Frequency-Aware Shadow Erasing Net](https://mlsanity.com/paper/2308.14221/) (2023)
- [Leveraging Inpainting for Single-Image Shadow Removal](https://mlsanity.com/paper/2302.05361/) (2023)
- [Recasting Regional Lighting for Shadow Removal](https://mlsanity.com/paper/2402.00341/) (2024)
- [Estimating Reflectance Layer from A Single Image: Integrating Reflectance Guidance and Shadow/Specular Aware Learning](https://mlsanity.com/paper/2211.14751/) (2022)
- [CRFormer: A Cross-Region Transformer for Shadow Removal](https://mlsanity.com/paper/2207.01600/) (2022)
- [Assessing Sample Quality via the Latent Space of Generative Models](https://mlsanity.com/paper/2407.15171/) (2024)
- [Temporal Feature Warping for Video Shadow Detection](https://mlsanity.com/paper/2107.14287/) (2021)
- [CNSNet: A Cleanness-Navigated-Shadow Network for Shadow Removal](https://mlsanity.com/paper/2209.02174/) (2022)
- [LP-IOANet: Efficient High Resolution Document Shadow Removal](https://mlsanity.com/paper/2303.12862/) (2023)
- [Shadow Removal Refinement via Material-Consistent Shadow Edges](https://mlsanity.com/paper/2409.06848/) (2024)
- [FieldNet: Efficient Real-Time Shadow Removal for Enhanced Vision in Field Robotics](https://mlsanity.com/paper/2403.08142/) (2024)
- [ShadowHack: Hacking Shadows via Luminance-Color Divide and Conquer](https://mlsanity.com/paper/2412.02545/) (2024)
- [After the Party: Navigating the Mapping From Color to Ambient Lighting](https://mlsanity.com/paper/2508.02168/) (2025)
- [Structure-Informed Shadow Removal Networks](https://mlsanity.com/paper/2301.03182/) (2023)
- [Learning Physical-Spatio-Temporal Features for Video Shadow Removal](https://mlsanity.com/paper/2303.09370/) (2023)
- [Progressive Recurrent Network for Shadow Removal](https://mlsanity.com/paper/2311.00455/) (2023)
- [Semantic-guided Adversarial Diffusion Model for Self-supervised Shadow Removal](https://mlsanity.com/paper/2407.01104/) (2024)
- [Fisheye Camera and Ultrasonic Sensor Fusion For Near-Field Obstacle Perception in Bird's-Eye-View](https://mlsanity.com/paper/2402.00637/) (2024)
- [Cross-Modal Spherical Aggregation for Weakly Supervised Remote Sensing Shadow Removal](https://mlsanity.com/paper/2406.17469/) (2024)
- [ShadowMamba: State-Space Model with Boundary-Region Selective Scan for Shadow Removal](https://mlsanity.com/paper/2411.03260/) (2024)
- [Controlling the Latent Diffusion Model for Generative Image Shadow Removal via Residual Generation](https://mlsanity.com/paper/2412.02322/) (2024)
- [NTIRE 2025 Image Shadow Removal Challenge Report](https://mlsanity.com/paper/2506.15524/) (2025)
- [DeshadowMamba: Deshadowing as 1D Sequential Similarity](https://mlsanity.com/paper/2510.24260/) (2025)
- [CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation](https://mlsanity.com/paper/2511.17755/) (2025)
- [UnShadowNet: Illumination Critic Guided Contrastive Learning For Shadow Removal](https://mlsanity.com/paper/2203.15441/) (2022)
- [Soft-Hard Attention U-Net Model and Benchmark Dataset for Multiscale Image Shadow Removal](https://mlsanity.com/paper/2408.03734/) (2024)

## Abstract (arXiv)

We propose a novel deep learning method for shadow removal. Inspired by physical models of shadow formation, we use a linear illumination transformation to model the shadow effects in the image that allows the shadow image to be expressed as a combination of the shadow-free image, the shadow parameters, and a matte layer. We use two deep networks, namely SP-Net and M-Net, to predict the shadow parameters and the shadow matte respectively. This system allows us to remove the shadow effects from images. We then employ an inpainting network, I-Net, to further refine the results. We train and test our framework on the most challenging shadow removal dataset (ISTD). Our method improves the state-of-the-art in terms of root mean square error (RMSE) for the shadow area by 20%. Furthermore, this decomposition allows us to formulate a patch-based weakly-supervised shadow removal method. This model can be trained without any shadow-free images (that are cumbersome to acquire) and achieves competitive shadow removal results compared to state-of-the-art methods that are trained with fully paired shadow and shadow-free images. Last, we introduce SBU-Timelapse, a video shadow removal dataset for evaluating shadow removal methods.