mlsanityData to 14 Oct 2026

STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation

arXiv 2202.03747 ↗First posted 8 Feb 2022cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Video Instance Segmentation (VIS) is a task that simultaneously requires classification, segmentation, and instance association in a video. Recent VIS approaches rely on sophisticated pipelines to achieve this goal, including RoI-related operations or 3D convolutions. In contrast, we present a simple and efficient single-stage VIS framework based on the instance segmentation method CondInst by adding an extra tracking head. To improve instance association accuracy, a novel bi-directional spatio-temporal contrastive learning strategy for tracking embedding across frames is proposed. Moreover, an instance-wise temporal consistency scheme is utilized to produce temporally coherent results. Experiments conducted on the YouTube-VIS-2019, YouTube-VIS-2021, and OVIS-2021 datasets validate the effectiveness and efficiency of the proposed method. We hope the proposed framework can serve as a simple and strong alternative for many other instance-level video association tasks.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 47 of its 56 references that are in the map. First posted on arXiv 8 Feb 2022.

Builds on

47 of 56 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2017Mask R-CNNcited by 7,277
  8. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  10. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  11. 2017Deformable Convolutional Networkscited by 1,488
  12. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
35 more
  1. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  2. 2018Path Aggregation Network for Instance Segmentationcited by 798
  3. 2019Hybrid Task Cascade for Instance Segmentationcited by 481
  4. 2019YOLACT: Real-time Instance Segmentationcited by 443
  5. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  6. 2020Hard Negative Mixing for Contrastive Learningcited by 402
  7. 2019Video Instance Segmentationcited by 393
  8. 2016Dynamic Filter Networkscited by 319
  9. 2020Conditional Convolutions for Instance Segmentationcited by 314
  10. 2019SOLO: Segmenting Objects by Locationscited by 289
  11. 2019Mask Scoring R-CNNcited by 266
  12. 2020Quasi-Dense Similarity Learning for Multiple Object Trackingcited by 220
  13. 2019PolarMask: Single Shot Instance Segmentation with Polar Representationcited by 188
  14. 2019FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentationcited by 178
  15. 2017Flow-Guided Feature Aggregation for Video Object Detectioncited by 177
  16. 2021Instances as Queriescited by 158
  17. 2021Track to Detect and Segment: An Online Multi-Object Trackercited by 156
  18. 2020BlendMask: Top-Down Meets Bottom-Up for Instance Segmentationcited by 145
  19. 2019Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagationcited by 125
  20. 2020STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videoscited by 121
  21. 2020SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentationcited by 108
  22. 2021Video Instance Segmentation using Inter-Frame Communication Transformerscited by 105
  23. 2018Efficient Video Object Segmentation via Network Modulationcited by 98
  24. 2019Sequence Level Semantics Aggregation for Video Object Detectioncited by 87
  25. 2021SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentationcited by 87
  26. 2021Crossover Learning for Fast Online Video Instance Segmentationcited by 83
  27. 2021Video Instance Segmentation with a Propose-Reduce Paradigmcited by 74
  28. 2018Object Detection in Video with Spatiotemporal Sampling Networkscited by 57
  29. 2020LoCo: Local Contrastive Representation Learningcited by 55
  30. 2020CompFeat: Comprehensive Feature Aggregation for Video Instance Segmentationcited by 53
  31. 2021Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentationcited by 43
  32. 2021Spatial Feature Calibration and Temporal Fusion for Effective One-stage Video Instance Segmentationcited by 42
  33. 2020CenterMask: single shot instance segmentation with point representationcited by 31
  34. 2021Rank & Sort Loss for Object Detection and Instance Segmentationcited by 22
  35. 2016Bidirectional Multirate Reconstruction for Temporal Modeling in Videoscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 0; 2025: 3; 2026: 2.

13 papers.

Sort:

  1. 2023Personalize Segment Anything Model with One Shotcited by 250
  2. 2022In Defense of Online Models for Video Instance Segmentationcited by 97
  3. 2022MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Trainingcited by 94
  4. 2023CTVIS: Consistent Training for Online Video Instance Segmentationcited by 51
  5. 2022Prototypical Contrast Adaptation for Domain Adaptive Semantic Segmentationcited by 40
  6. 2023Rehabilitation Exercise Repetition Segmentation and Counting using Skeletal Body Jointscited by 3
  7. 2023Prompt-Driven Building Footprint Extraction in Aerial Images with Offset-Building Modelcited by 3
  8. 2025A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospectscited by 2
  9. 2023Dual Path Transformer with Partition Attentioncited by 1
  10. 2025Local2Global query Alignment for Video Instance Segmentationcited by 0
  11. 2025LTCA: Long-range Temporal Context Attention for Referring Video Object Segmentationcited by 0
  12. 2026Structured prototype regularization for synthetic-to-real driving scene parsingcited by 0
1 more
  1. 2026SA-VIS: Sparse frame Annotations for training Video Instance Segmentationcited by 0