# ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal

arXiv: https://arxiv.org/abs/2212.04711 · HTML: https://mlsanity.com/paper/2212.04711/

## Key facts

- Cited by 103 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 46 of its 84 references that are in the map.
- First posted on arXiv 9 Dec 2022.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement](https://mlsanity.com/paper/2001.06826/) (2020)
- [EnlightenGAN: Deep Light Enhancement without Paired Supervision](https://mlsanity.com/paper/1906.06972/) (2019)
- [Kindling the Darkness: A Practical Low-light Image Enhancer](https://mlsanity.com/paper/1905.04161/) (2019)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Learning Enriched Features for Real Image Restoration and Enhancement](https://mlsanity.com/paper/2003.06792/) (2020)
- [Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement](https://mlsanity.com/paper/2012.05609/) (2020)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction](https://mlsanity.com/paper/1704.02422/) (2017)
- [Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models](https://mlsanity.com/paper/2207.14626/) (2022)
- [SRFlow: Learning the Super-Resolution Space with Normalizing Flow](https://mlsanity.com/paper/2006.14200/) (2020)
- [Low-Light Image Enhancement with Normalizing Flow](https://mlsanity.com/paper/2109.05923/) (2021)
- [HDR image reconstruction from a single exposure using deep CNNs](https://mlsanity.com/paper/1710.07480/) (2017)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks](https://mlsanity.com/paper/1704.02470/) (2017)
- [Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN](https://mlsanity.com/paper/1911.08718/) (2019)

## Built on by (most cited first, 30 of 103)

- [Diffusion Model-Based Image Editing: A Survey](https://mlsanity.com/paper/2402.17525/) (2024)
- [Low-Light Image Enhancement with Wavelet-based Diffusion Models](https://mlsanity.com/paper/2306.00306/) (2023)
- [Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2308.09388/) (2023)
- [AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion](https://mlsanity.com/paper/2310.10123/) (2023)
- [ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion](https://mlsanity.com/paper/2403.18818/) (2024)
- [Residual Denoising Diffusion Models](https://mlsanity.com/paper/2308.13712/) (2023)
- [I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing](https://mlsanity.com/paper/2408.14180/) (2024)
- [DMPlug: A Plug-in Method for Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2405.16749/) (2024)
- [LightIt: Illumination Modeling and Control for Diffusion Models](https://mlsanity.com/paper/2403.10615/) (2024)
- [Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Tasks](https://mlsanity.com/paper/2403.00644/) (2024)
- [Backdoor Attacks Against Deep Image Compression via Adaptive Frequency Trigger](https://mlsanity.com/paper/2302.14677/) (2023)
- [Vision-Language Gradient Descent-driven All-in-One Deep Unfolding Networks](https://mlsanity.com/paper/2503.16930/) (2025)
- [ShadowFormer: Global Context Helps Image Shadow Removal](https://mlsanity.com/paper/2302.01650/) (2023)
- [Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving](https://mlsanity.com/paper/2404.04804/) (2024)
- [ObjectMover: Generative Object Movement with Video Prior](https://mlsanity.com/paper/2503.08037/) (2025)
- [ExposureDiffusion: Learning to Expose for Low-light Image Enhancement](https://mlsanity.com/paper/2307.07710/) (2023)
- [Latent Feature-Guided Diffusion Models for Shadow Removal](https://mlsanity.com/paper/2312.02156/) (2023)
- [Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint](https://mlsanity.com/paper/2409.15739/) (2024)
- [RORem: Training a Robust Object Remover with Human-in-the-Loop](https://mlsanity.com/paper/2501.00740/) (2025)
- [UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video](https://mlsanity.com/paper/2306.09349/) (2023)
- [Towards Real-World Blind Face Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2312.15736/) (2023)
- [AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising](https://mlsanity.com/paper/2406.06911/) (2024)
- [Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise](https://mlsanity.com/paper/2311.14900/) (2023)
- [OmniSR: Shadow Removal under Direct and Indirect Lighting](https://mlsanity.com/paper/2410.01719/) (2024)
- [QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity](https://mlsanity.com/paper/2212.10431/) (2022)
- [Raw Image Reconstruction with Learned Compact Metadata](https://mlsanity.com/paper/2302.12995/) (2023)
- [Shadow Generation for Composite Image Using Diffusion model](https://mlsanity.com/paper/2403.15234/) (2024)
- [LLDiffusion: Learning Degradation Representations in Diffusion Models for Low-Light Image Enhancement](https://mlsanity.com/paper/2307.14659/) (2023)
- [Improving the Stability and Efficiency of Diffusion Models for Content Consistent Super-Resolution](https://mlsanity.com/paper/2401.00877/) (2023)
- [DP-IQA: Utilizing Diffusion Prior for Blind Image Quality Assessment in the Wild](https://mlsanity.com/paper/2405.19996/) (2024)

## Abstract (arXiv)

Recent deep learning methods have achieved promising results in image shadow removal. However, their restored images still suffer from unsatisfactory boundary artifacts, due to the lack of degradation prior embedding and the deficiency in modeling capacity. Our work addresses these issues by proposing a unified diffusion framework that integrates both the image and degradation priors for highly effective shadow removal. In detail, we first propose a shadow degradation model, which inspires us to build a novel unrolling diffusion model, dubbed ShandowDiffusion. It remarkably improves the model's capacity in shadow removal via progressively refining the desired output with both degradation prior and diffusive generative prior, which by nature can serve as a new strong baseline for image restoration. Furthermore, ShadowDiffusion progressively refines the estimated shadow mask as an auxiliary task of the diffusion generator, which leads to more accurate and robust shadow-free image generation. We conduct extensive experiments on three popular public datasets, including ISTD, ISTD+, and SRD, to validate our method's effectiveness. Compared to the state-of-the-art methods, our model achieves a significant improvement in terms of PSNR, increasing from 31.69dB to 34.73dB over SRD dataset.