# Instance Shadow Detection

arXiv: https://arxiv.org/abs/1911.07034 · HTML: https://mlsanity.com/paper/1911.07034/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2019.

## Built on by (most cited first, 30 of 37)

- [ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion](https://mlsanity.com/paper/2403.18818/) (2024)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [From Shadow Generation to Shadow Removal](https://mlsanity.com/paper/2103.12997/) (2021)
- [Omnimatte: Associating Objects and Their Effects in Video](https://mlsanity.com/paper/2105.06993/) (2021)
- [Shadow Generation for Composite Image in Real-world Scenes](https://mlsanity.com/paper/2104.10338/) (2021)
- [A General Protocol to Probe Large Vision Models for 3D Physical Understanding](https://mlsanity.com/paper/2310.06836/) (2023)
- [ObjectMover: Generative Object Movement with Video Prior](https://mlsanity.com/paper/2503.08037/) (2025)
- [Latent Feature-Guided Diffusion Models for Shadow Removal](https://mlsanity.com/paper/2312.02156/) (2023)
- [Learning from Synthetic Shadows for Shadow Detection and Removal](https://mlsanity.com/paper/2101.01713/) (2021)
- [Instance Shadow Detection with A Single-Stage Detector](https://mlsanity.com/paper/2207.04614/) (2022)
- [SCOTCH and SODA: A Transformer Video Shadow Detection Framework](https://mlsanity.com/paper/2211.06885/) (2022)
- [SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels](https://mlsanity.com/paper/2308.12064/) (2023)
- [DepthCues: Evaluating Monocular Depth Perception in Large Vision Models](https://mlsanity.com/paper/2411.17385/) (2024)
- [SIMBAR: Single Image-Based Scene Relighting For Effective Data Augmentation For Automated Driving Vision Tasks](https://mlsanity.com/paper/2204.00644/) (2022)
- [OmnimatteRF: Robust Omnimatte with 3D Background Modeling](https://mlsanity.com/paper/2309.07749/) (2023)
- [Casual Indoor HDR Radiance Capture from Omnidirectional Images](https://mlsanity.com/paper/2208.07903/) (2022)
- [SDDNet: Style-guided Dual-layer Disentanglement Network for Shadow Detection](https://mlsanity.com/paper/2308.08935/) (2023)
- [Learning Shadow Correspondence for Video Shadow Detection](https://mlsanity.com/paper/2208.00150/) (2022)
- [Shadows Shed Light on 3D Objects](https://mlsanity.com/paper/2206.08990/) (2022)
- [Language-Driven Interactive Shadow Detection](https://mlsanity.com/paper/2408.08543/) (2024)
- [Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Era](https://mlsanity.com/paper/2409.02108/) (2024)
- [S3R-Net: A Single-Stage Approach to Self-Supervised Shadow Removal](https://mlsanity.com/paper/2404.12103/) (2024)
- [Inferring Dynamic Physical Properties from Video Foundation Models](https://mlsanity.com/paper/2510.02311/) (2025)
- [AdapterShadow: Adapting Segment Anything Model for Shadow Detection](https://mlsanity.com/paper/2311.08891/) (2023)
- [Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation](https://mlsanity.com/paper/2409.02567/) (2024)
- [MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesis](https://mlsanity.com/paper/2412.02635/) (2024)
- [Video Instance Shadow Detection Under the Sun and Sky](https://mlsanity.com/paper/2211.12827/) (2022)
- [Prompt-Aware Controllable Shadow Removal](https://mlsanity.com/paper/2501.15043/) (2025)
- [ClueAegis: Heuristic-to-Reasoning Cognitive-skill Learning for Unified Evidence-based Synthetic Image Detection](https://mlsanity.com/paper/2605.25009/) (2026)
- [UnShadowNet: Illumination Critic Guided Contrastive Learning For Shadow Removal](https://mlsanity.com/paper/2203.15441/) (2022)

## Abstract (arXiv)

Instance shadow detection is a brand new problem, aiming to find shadow instances paired with object instances. To approach it, we first prepare a new dataset called SOBA, named after Shadow-OBject Association, with 3,623 pairs of shadow and object instances in 1,000 photos, each with individual labeled masks. Second, we design LISA, named after Light-guided Instance Shadow-object Association, an end-to-end framework to automatically predict the shadow and object instances, together with the shadow-object associations and light direction. Then, we pair up the predicted shadow and object instances, and match them with the predicted shadow-object associations to generate the final results. In our evaluations, we formulate a new metric named the shadow-object average precision to measure the performance of our results. Further, we conducted various experiments and demonstrate our method's applicability on light direction estimation and photo editing.