mlsanityData to 7 Oct 2026

Video Instance Shadow Detection Under the Sun and Sky

arXiv 2211.12827 ↗First posted 23 Nov 2022cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Instance shadow detection, crucial for applications such as photo editing and light direction estimation, has undergone significant advancements in predicting shadow instances, object instances, and their associations. The extension of this task to videos presents challenges in annotating diverse video data and addressing complexities arising from occlusion and temporary disappearances within associations. In response to these challenges, we introduce ViShadow, a semi-supervised video instance shadow detection framework that leverages both labeled image data and unlabeled video data for training. ViShadow features a two-stage training pipeline: the first stage, utilizing labeled image data, identifies shadow and object instances through contrastive learning for cross-frame pairing. The second stage employs unlabeled videos, incorporating an associated cycle consistency loss to enhance tracking ability. A retrieval mechanism is introduced to manage temporary disappearances, ensuring tracking continuity. The SOBA-VID dataset, comprising unlabeled training videos and labeled testing videos, along with the SOAP-VID metric, is introduced for the quantitative evaluation of VISD solutions. The effectiveness of ViShadow is further demonstrated through various video-level applications such as video inpainting, instance cloning, shadow editing, and text-instructed shadow-object manipulation.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 32 of its 59 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

32 of 59 references.

References · 32

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  7. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  8. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  9. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  10. 2019Video Instance Segmentationcited by 391
  11. 2020Conditional Convolutions for Instance Segmentationcited by 314
  12. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
20 more
  1. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  2. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  3. 2022Towards An End-to-End Framework for Flow-Guided Video Inpaintingcited by 111
  4. 2021Video Instance Segmentation using Inter-Frame Communication Transformerscited by 105
  5. 2021SeqFormer: Sequential Transformer for Video Instance Segmentationcited by 98
  6. 2021FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpaintingcited by 94
  7. 2019Joint-task Self-supervised Learning for Temporal Correspondencecited by 80
  8. 2018Direction-aware Spatial Context Features for Shadow Detection and Removalcited by 58
  9. 2017Direction-aware Spatial Context Features for Shadow Detectioncited by 56
  10. 2021Omnimatte: Associating Objects and Their Effects in Videocited by 50
  11. 2018The 2018 DAVIS Challenge on Video Object Segmentationcited by 47
  12. 2017A+D Net: Training a Shadow Detector with Adversarial Shadow Attenuationcited by 39
  13. 2019Instance Shadow Detectioncited by 37
  14. 2019Revisiting Shadow Detection: A New Benchmark Dataset for Complex Worldcited by 34
  15. 2021Triple-cooperative Video Shadow Detectioncited by 22
  16. 2022Instance Shadow Detection with A Single-Stage Detectorcited by 22
  17. 2022SCOTCH and SODA: A Transformer Video Shadow Detection Frameworkcited by 19
  18. 2021Learning to Track Instances without Video Annotationscited by 18
  19. 2022Video Shadow Detection via Spatio-Temporal Interpolation Consistency Trainingcited by 12
  20. 2022Learning Shadow Correspondence for Video Shadow Detectioncited by 9

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2024RenDetNet: Weakly-supervised Shadow Detection with Shadow Caster Verificationcited by 0
  2. 2025FastInstShadow: A Simple Query-Based Model for Instance Shadow Detectioncited by 0
  3. 2025Cast and Attached Shadow Detection via Iterative Light and Geometry Reasoningcited by 0