# Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN

arXiv: https://arxiv.org/abs/1911.08718 · HTML: https://mlsanity.com/paper/1911.08718/

## Key facts

- Cited by 103 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2019.

## Built on by (most cited first, 30 of 103)

- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Explicit Visual Prompting for Low-Level Structure Segmentations](https://mlsanity.com/paper/2303.10883/) (2023)
- [ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal](https://mlsanity.com/paper/2212.04711/) (2022)
- [Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Models](https://mlsanity.com/paper/2304.08291/) (2023)
- [Auto-Exposure Fusion for Single-Image Shadow Removal](https://mlsanity.com/paper/2103.01255/) (2021)
- [Spatially-Adaptive Image Restoration using Distortion-Guided Networks](https://mlsanity.com/paper/2108.08617/) (2021)
- [ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion](https://mlsanity.com/paper/2403.18818/) (2024)
- [DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network](https://mlsanity.com/paper/2207.10434/) (2022)
- [Residual Denoising Diffusion Models](https://mlsanity.com/paper/2308.13712/) (2023)
- [SAM2-Adapter: Evaluating & Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and More](https://mlsanity.com/paper/2408.04579/) (2024)
- [TAPE: Task-Agnostic Prior Embedding for Image Restoration](https://mlsanity.com/paper/2203.06074/) (2022)
- [SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More](https://mlsanity.com/paper/2304.09148/) (2023)
- [CANet: A Context-Aware Network for Shadow Removal](https://mlsanity.com/paper/2108.09894/) (2021)
- [DeS3: Adaptive Attention-driven Self and Soft Shadow Removal using ViT Similarity](https://mlsanity.com/paper/2211.08089/) (2022)
- [High-Resolution Document Shadow Removal via A Large-Scale Real-World Dataset and A Frequency-Aware Shadow Erasing Net](https://mlsanity.com/paper/2308.14221/) (2023)
- [Shadow Generation for Composite Image in Real-world Scenes](https://mlsanity.com/paper/2104.10338/) (2021)
- [ShadowFormer: Global Context Helps Image Shadow Removal](https://mlsanity.com/paper/2302.01650/) (2023)
- [Latent Feature-Guided Diffusion Models for Shadow Removal](https://mlsanity.com/paper/2312.02156/) (2023)
- [Learning from Synthetic Shadows for Shadow Detection and Removal](https://mlsanity.com/paper/2101.01713/) (2021)
- [Leveraging Inpainting for Single-Image Shadow Removal](https://mlsanity.com/paper/2302.05361/) (2023)
- [Instance Shadow Detection with A Single-Stage Detector](https://mlsanity.com/paper/2207.04614/) (2022)
- [A Large-scale Film Style Dataset for Learning Multi-frequency Driven Film Enhancement](https://mlsanity.com/paper/2301.08880/) (2023)
- [ShadowRefiner: Towards Mask-free Shadow Removal via Fast Fourier Transformer](https://mlsanity.com/paper/2406.02559/) (2024)
- [Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise](https://mlsanity.com/paper/2311.14900/) (2023)
- [OmniSR: Shadow Removal under Direct and Indirect Lighting](https://mlsanity.com/paper/2410.01719/) (2024)
- [Blind Image Decomposition](https://mlsanity.com/paper/2108.11364/) (2021)
- [Explicit Visual Prompting for Universal Foreground Segmentations](https://mlsanity.com/paper/2305.18476/) (2023)
- [Devignet: High-Resolution Vignetting Removal via a Dual Aggregated Fusion Transformer With Adaptive Channel Expansion](https://mlsanity.com/paper/2308.13739/) (2023)
- [Unsupervised Portrait Shadow Removal via Generative Priors](https://mlsanity.com/paper/2108.03466/) (2021)
- [ShaDocNet: Learning Spatial-Aware Tokens in Transformer for Document Shadow Removal](https://mlsanity.com/paper/2211.16675/) (2022)

## Abstract (arXiv)

Shadow removal is an essential task for scene understanding. Many studies consider only matching the image contents, which often causes two types of ghosts: color in-consistencies in shadow regions or artifacts on shadow boundaries. In this paper, we tackle these issues in two ways. First, to carefully learn the border artifacts-free image, we propose a novel network structure named the dual hierarchically aggregation network~(DHAN). It contains a series of growth dilated convolutions as the backbone without any down-samplings, and we hierarchically aggregate multi-context features for attention and prediction, respectively. Second, we argue that training on a limited dataset restricts the textural understanding of the network, which leads to the shadow region color in-consistencies. Currently, the largest dataset contains 2k+ shadow/shadow-free image pairs. However, it has only 0.1k+ unique scenes since many samples share exactly the same background with different shadow positions. Thus, we design a shadow matting generative adversarial network~(SMGAN) to synthesize realistic shadow mattings from a given shadow mask and shadow-free image. With the help of novel masks or scenes, we enhance the current datasets using synthesized shadow images. Experiments show that our DHAN can erase the shadows and produce high-quality ghost-free images. After training on the synthesized and real datasets, our network outperforms other state-of-the-art methods by a large margin. The code is available: http://github.com/vinthony/ghost-free-shadow-removal/