# RenDetNet: Weakly-supervised Shadow Detection with Shadow Caster Verification

arXiv: https://arxiv.org/abs/2408.17143 · HTML: https://mlsanity.com/paper/2408.17143/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 49 references that are in the map.
- First posted on arXiv 30 Aug 2024.

## Builds on (most cited first, 27 of 27)

- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [AGORA: Avatars in Geography Optimized for Regression Analysis](https://mlsanity.com/paper/2104.14643/) (2021)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects](https://mlsanity.com/paper/1904.03167/) (2019)
- [Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data](https://mlsanity.com/paper/1903.10683/) (2019)
- [Bridging Composite and Real: Towards End-to-end Deep Image Matting](https://mlsanity.com/paper/2010.16188/) (2020)
- [DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network](https://mlsanity.com/paper/2207.10434/) (2022)
- [Direction-aware Spatial Context Features for Shadow Detection](https://mlsanity.com/paper/1712.04142/) (2017)
- [Portrait Shadow Manipulation](https://mlsanity.com/paper/2005.08925/) (2020)
- [A+D Net: Training a Shadow Detector with Adversarial Shadow Attenuation](https://mlsanity.com/paper/1712.01361/) (2017)
- [Shadow Removal by a Lightness-Guided Network with Training on Unpaired Data](https://mlsanity.com/paper/2006.15617/) (2020)
- [Instance Shadow Detection](https://mlsanity.com/paper/1911.07034/) (2019)
- [Learning from Synthetic Shadows for Shadow Detection and Removal](https://mlsanity.com/paper/2101.01713/) (2021)
- [Instance Shadow Detection with A Single-Stage Detector](https://mlsanity.com/paper/2207.04614/) (2022)
- [SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels](https://mlsanity.com/paper/2308.12064/) (2023)
- [Water-Filling: An Efficient Algorithm for Digitized Document Shadow Removal](https://mlsanity.com/paper/1904.09763/) (2019)
- [Multi-View Fusion for Multi-Level Robotic Scene Understanding](https://mlsanity.com/paper/2103.13539/) (2021)
- [Video Shadow Detection via Spatio-Temporal Interpolation Consistency Training](https://mlsanity.com/paper/2206.08801/) (2022)
- [SDDNet: Style-guided Dual-layer Disentanglement Network for Shadow Detection](https://mlsanity.com/paper/2308.08935/) (2023)
- [Multi-View Consistency Loss for Improved Single-Image 3D Reconstruction of Clothed People](https://mlsanity.com/paper/2009.14162/) (2020)
- [Fine-Context Shadow Detection using Shadow Removal](https://mlsanity.com/paper/2109.09609/) (2021)
- [S3R-Net: A Single-Stage Approach to Self-Supervised Shadow Removal](https://mlsanity.com/paper/2404.12103/) (2024)
- [Video Instance Shadow Detection Under the Sun and Sky](https://mlsanity.com/paper/2211.12827/) (2022)
- [Shadow Detection for Ultrasound Images Using Unlabeled Data and Synthetic Shadows](https://mlsanity.com/paper/1908.01439/) (2019)
- [Shadow-Oriented Tracking Method for Multi-Target Tracking in Video-SAR](https://mlsanity.com/paper/2211.15995/) (2022)
- [AI driven shadow model detection in agropv farms](https://mlsanity.com/paper/2304.07853/) (2023)

## Abstract (arXiv)

Existing shadow detection models struggle to differentiate dark image areas from shadows. In this paper, we tackle this issue by verifying that all detected shadows are real, i.e. they have paired shadow casters. We perform this step in a physically-accurate manner by differentiably re-rendering the scene and observing the changes stemming from carving out estimated shadow casters. Thanks to this approach, the RenDetNet proposed in this paper is the first learning-based shadow detection model whose supervisory signals can be computed in a self-supervised manner. The developed system compares favourably against recent models trained on our data. As part of this publication, we release our code on github.