# ShadowMaskFormer: Mask Augmented Patch Embeddings for Shadow Removal

arXiv: https://arxiv.org/abs/2404.18433 · HTML: https://mlsanity.com/paper/2404.18433/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 54 references that are in the map.
- First posted on arXiv 29 Apr 2024.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Vision Transformers for Single Image Dehazing](https://mlsanity.com/paper/2204.03883/) (2022)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [General Multi-label Image Classification with Transformers](https://mlsanity.com/paper/2011.14027/) (2020)
- [Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN](https://mlsanity.com/paper/1911.08718/) (2019)
- [ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal](https://mlsanity.com/paper/2212.04711/) (2022)
- [Shadow Removal via Shadow Image Decomposition](https://mlsanity.com/paper/1908.08628/) (2019)
- [Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data](https://mlsanity.com/paper/1903.10683/) (2019)
- [Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding](https://mlsanity.com/paper/2203.00867/) (2022)
- [Auto-Exposure Fusion for Single-Image Shadow Removal](https://mlsanity.com/paper/2103.01255/) (2021)
- [DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network](https://mlsanity.com/paper/2207.10434/) (2022)
- [From Shadow Segmentation to Shadow Removal](https://mlsanity.com/paper/2008.00267/) (2020)
- [Direction-aware Spatial Context Features for Shadow Detection and Removal](https://mlsanity.com/paper/1805.04635/) (2018)
- [From Shadow Generation to Shadow Removal](https://mlsanity.com/paper/2103.12997/) (2021)
- [ARGAN: Attentive Recurrent Generative Adversarial Network for Shadow Detection and Removal](https://mlsanity.com/paper/1908.01323/) (2019)
- [CANet: A Context-Aware Network for Shadow Removal](https://mlsanity.com/paper/2108.09894/) (2021)
- [RIS-GAN: Explore Residual and Illumination with Generative Adversarial Networks for Shadow Removal](https://mlsanity.com/paper/1911.09178/) (2019)
- [Improving Shadow Suppression for Illumination Robust Face Recognition](https://mlsanity.com/paper/1710.05073/) (2017)
- [Leveraging Inpainting for Single-Image Shadow Removal](https://mlsanity.com/paper/2302.05361/) (2023)
- [Shadow-Aware Dynamic Convolution for Shadow Removal](https://mlsanity.com/paper/2205.04908/) (2022)
- [Semi-supervised Deep Learning for Image Classification with Distribution Mismatch: A Survey](https://mlsanity.com/paper/2203.00190/) (2022)
- [CRFormer: A Cross-Region Transformer for Shadow Removal](https://mlsanity.com/paper/2207.01600/) (2022)

## Built on by (most cited first, 7 of 7)

- [Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Era](https://mlsanity.com/paper/2409.02108/) (2024)
- [ShadowGS: Shadow-Aware 3D Gaussian Splatting for Satellite Imagery](https://mlsanity.com/paper/2601.00939/) (2026)
- [ShadowMamba: State-Space Model with Boundary-Region Selective Scan for Shadow Removal](https://mlsanity.com/paper/2411.03260/) (2024)
- [DeshadowMamba: Deshadowing as 1D Sequential Similarity](https://mlsanity.com/paper/2510.24260/) (2025)
- [FSSUWNet: Mitigating the Fragility of Pre-trained Models with Feature Enhancement for Few-Shot Semantic Segmentation in Underwater Images](https://mlsanity.com/paper/2504.00478/) (2025)
- [D2-Mamba: Dual-Scale Fusion and Dual-Path Scanning with SSMs for Shadow Removal](https://mlsanity.com/paper/2508.12750/) (2025)
- [OmniLight: One Model to Rule All Lighting Conditions](https://mlsanity.com/paper/2604.15170/) (2026)

## Abstract (arXiv)

Transformer recently emerged as the de facto model for computer vision tasks and has also been successfully applied to shadow removal. However, these existing methods heavily rely on intricate modifications to the attention mechanisms within the transformer blocks while using a generic patch embedding. As a result, it often leads to complex architectural designs requiring additional computation resources. In this work, we aim to explore the efficacy of incorporating shadow information within the early processing stage. Accordingly, we propose a transformer-based framework with a novel patch embedding that is tailored for shadow removal, dubbed ShadowMaskFormer. Specifically, we present a simple and effective mask-augmented patch embedding to integrate shadow information and promote the model's emphasis on acquiring knowledge for shadow regions. Extensive experiments conducted on the ISTD, ISTD+, and SRD benchmark datasets demonstrate the efficacy of our method against state-of-the-art approaches while using fewer model parameters.g fewer model parameters. Our implementation is available at https://github.com/lizhh268/ShadowMaskFormer.