mlsanityData to 14 Oct 2026

SCOTCH and SODA: A Transformer Video Shadow Detection Framework

arXiv 2211.06885 ↗First posted 13 Nov 2022cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Shadows in videos are difficult to detect because of the large shadow deformation between frames. In this work, we argue that accounting for shadow deformation is essential when designing a video shadow detection method. To this end, we introduce the shadow deformation attention trajectory (SODA), a new type of video self-attention module, specially designed to handle the large shadow deformations in videos. Moreover, we present a new shadow contrastive learning mechanism (SCOTCH) which aims at guiding the network to learn a unified shadow representation from massive positive shadow pairs across different videos. We demonstrate empirically the effectiveness of our two contributions in an ablation study. Furthermore, we show that SCOTCH and SODA significantly outperforms existing techniques for video shadow detection. Code is available at the project page: https://lihaoliu-cambridge.github.io/scotch_and_soda/

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 32 of its 47 references that are in the map. First posted on arXiv 13 Nov 2022.

Builds on

32 of 47 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  7. 2016Pyramid Scene Parsing Networkcited by 2,479
  8. 2017Non-local Neural Networkscited by 2,293
  9. 2021ViViT: A Video Vision Transformercited by 1,620
  10. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  11. 2019Video Object Segmentation using Space-Time Memory Networkscited by 438
  12. 2019Interpolation Consistency Training for Semi-Supervised Learningcited by 251
20 more
  1. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  2. 2016Deeply supervised salient object detection with short connectionscited by 196
  3. 2019FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentationcited by 178
  4. 2015Delving Deeper into Convolutional Networks for Learning Video Representationscited by 148
  5. 2022Multiview Transformers for Video Recognitioncited by 148
  6. 2020See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networkscited by 138
  7. 2021VidTr: Video Transformer Without Convolutionscited by 112
  8. 2021Space-time Mixing Attention for Video Transformercited by 84
  9. 2019Motion Guided Attention for Video Salient Object Detectioncited by 72
  10. 2017Direction-aware Spatial Context Features for Shadow Detectioncited by 56
  11. 2017Artificial Intelligence and Statisticscited by 43
  12. 2019Instance Shadow Detectioncited by 37
  13. 2019Revisiting Shadow Detection: A New Benchmark Dataset for Complex Worldcited by 34
  14. 2021Triple-cooperative Video Shadow Detectioncited by 22
  15. 2020Contrastive Registration for Unsupervised Medical Image Segmentationcited by 19
  16. 2022Video Shadow Detection via Spatio-Temporal Interpolation Consistency Trainingcited by 12
  17. 2022Learning Shadow Correspondence for Video Shadow Detectioncited by 9
  18. 2022PC-SwinMorph: Patch Representation for Unsupervised Medical Image Registration and Segmentationcited by 8
  19. 2021Temporal Feature Warping for Video Shadow Detectioncited by 5
  20. 2022Simultaneous Semantic and Instance Segmentation for Colon Nuclei Identification and Countingcited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 5; 2024: 8; 2025: 5.

19 papers.

Sort:

By area:

  1. 2024Cross-conditioned Diffusion Model for Medical Image to Image Translationcited by 26
  2. 2025SegDINO: An Efficient Design for Medical and Natural Image Segmentation with DINO-V3cited by 20
  3. 2023Detect Any Shadow: Segment Anything for Video Shadow Detectioncited by 16
  4. 2024MAMBA4D: Efficient Long-Sequence Point Cloud Video Understanding with Disentangled Spatial-Temporal State Space Modelscited by 11
  5. 2025Controllable-LPMoE: Adapting to Challenging Object Segmentation via Dynamic Local Priors from Mixture-of-Expertscited by 9
  6. 2024Language-Driven Interactive Shadow Detectioncited by 7
  7. 2024Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Eracited by 7
  8. 2023Large Scale Foundation Models for Intelligent Manufacturing Applications: A Surveycited by 6
  9. 2024Timeline and Boundary Guided Diffusion Network for Video Shadow Detectioncited by 6
  10. 2024Segment Anything for Videos: A Systematic Surveycited by 5
  11. 2023CD-CTFM: A Lightweight CNN-Transformer Network for Remote Sensing Cloud Detection Fusing Multiscale Featurescited by 4
  12. 2023Traffic Video Object Detection using Motion Priorcited by 4
7 more
  1. 2023TrafficMOT: A Challenging Dataset for Multi-Object Tracking in Complex Traffic Scenarioscited by 4
  2. 2022Video Instance Shadow Detection Under the Sun and Skycited by 3
  3. 2024Optimised ProPainter for Video Diminished Reality Inpaintingcited by 0
  4. 2024Fusion of Short-term and Long-term Attention for Video Mirror Detectioncited by 0
  5. 2025MirrorSAM2: Segment Mirror in Videos with Depth Perceptioncited by 0
  6. 2025DTTNet: Improving Video Shadow Detection via Dark-Aware Guidance and Tokenized Temporal Modelingcited by 0
  7. 2025Rethinking Surgical Smoke: A Smoke-Type-Aware Laparoscopic Video Desmoking Method and Datasetcited by 0