# Prompt-Aware Controllable Shadow Removal

arXiv: https://arxiv.org/abs/2501.15043 · HTML: https://mlsanity.com/paper/2501.15043/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 58 references that are in the map.
- First posted on arXiv 25 Jan 2025.

## Builds on (most cited first, 30 of 31)

- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing](https://mlsanity.com/paper/2111.15666/) (2021)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal](https://mlsanity.com/paper/2212.04711/) (2022)
- [Shadow Removal via Shadow Image Decomposition](https://mlsanity.com/paper/1908.08628/) (2019)
- [Single-Model and Any-Modality for Video Object Tracking](https://mlsanity.com/paper/2311.15851/) (2023)
- [Direction-aware Spatial Context Features for Shadow Detection and Removal](https://mlsanity.com/paper/1805.04635/) (2018)
- [SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation](https://mlsanity.com/paper/2305.17011/) (2023)
- [A+D Net: Training a Shadow Detector with Adversarial Shadow Attenuation](https://mlsanity.com/paper/1712.01361/) (2017)
- [Instance Shadow Detection](https://mlsanity.com/paper/1911.07034/) (2019)
- [ShadowFormer: Global Context Helps Image Shadow Removal](https://mlsanity.com/paper/2302.01650/) (2023)
- [The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances](https://mlsanity.com/paper/2009.13290/) (2020)
- [Latent Feature-Guided Diffusion Models for Shadow Removal](https://mlsanity.com/paper/2312.02156/) (2023)
- [Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture](https://mlsanity.com/paper/2403.07347/) (2024)
- [Leveraging Inpainting for Single-Image Shadow Removal](https://mlsanity.com/paper/2302.05361/) (2023)
- [Recasting Regional Lighting for Shadow Removal](https://mlsanity.com/paper/2402.00341/) (2024)
- [Instance Shadow Detection with A Single-Stage Detector](https://mlsanity.com/paper/2207.04614/) (2022)
- [Prototypical Calibrating Ambiguous Samples for Micro-Action Recognition](https://mlsanity.com/paper/2412.14719/) (2024)
- [MMAD: Multi-label Micro-Action Detection in Videos](https://mlsanity.com/paper/2407.05311/) (2024)
- [SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels](https://mlsanity.com/paper/2308.12064/) (2023)
- [OmniSR: Shadow Removal under Direct and Indirect Lighting](https://mlsanity.com/paper/2410.01719/) (2024)
- [Shadow Generation for Composite Image Using Diffusion model](https://mlsanity.com/paper/2403.15234/) (2024)
- [Repetitive Action Counting with Hybrid Temporal Relation Modeling](https://mlsanity.com/paper/2412.07233/) (2024)
- [Regional Attention for Shadow Removal](https://mlsanity.com/paper/2411.14201/) (2024)
- [Shadow Optimization from Structured Deep Edge Detection](https://mlsanity.com/paper/1505.01589/) (2015)
- [Diff-Shadow: Global-guided Diffusion Model for Shadow Removal](https://mlsanity.com/paper/2407.16214/) (2024)
- [Semi-Supervised Video Inpainting with Cycle Consistency Constraints](https://mlsanity.com/paper/2208.06807/) (2022)
- [SoftShadow: Leveraging Soft Masks for Penumbra-Aware Shadow Removal](https://mlsanity.com/paper/2409.07041/) (2024)

## Built on by (most cited first, 2 of 2)

- [BVINet: Unlocking Blind Video Inpainting with Zero Annotations](https://mlsanity.com/paper/2502.01181/) (2025)
- [ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generation](https://mlsanity.com/paper/2512.02453/) (2025)

## Abstract (arXiv)

Shadow removal aims to restore the image content in shadowed regions. While deep learning-based methods have shown promising results, they still face key challenges: 1) uncontrolled removal of all shadows, or 2) controllable removal but heavily relies on precise shadow region masks. To address these issues, we introduce a novel paradigm: prompt-aware controllable shadow removal. Unlike existing approaches, our paradigm allows for targeted shadow removal from specific subjects based on user prompts (e.g., dots, lines, or subject masks). This approach eliminates the need for shadow annotations and offers flexible, user-controlled shadow removal. Specifically, we propose an end-to-end learnable model, the Prompt-Aware Controllable Shadow Removal Network (PACSRNet). PACSRNet consists of two key modules: a prompt-aware module that generates shadow masks for the specified subject based on the user prompt, and a shadow removal module that uses the shadow prior from the first module to restore the content in the shadowed regions. Additionally, we enhance the shadow removal module by incorporating feature information from the prompt-aware module through a linear operation, providing prompt-guided support for shadow removal. Recognizing that existing shadow removal datasets lack diverse user prompts, we contribute a new dataset specifically designed for prompt-based controllable shadow removal. Extensive experimental results demonstrate the effectiveness and superiority of PACSRNet.