# STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation

arXiv: https://arxiv.org/abs/2202.03747 · HTML: https://mlsanity.com/paper/2202.03747/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 56 references that are in the map.
- First posted on arXiv 8 Feb 2022.

## Builds on (most cited first, 30 of 47)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [Dynamic Filter Networks](https://mlsanity.com/paper/1605.09673/) (2016)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020)
- [PolarMask: Single Shot Instance Segmentation with Polar Representation](https://mlsanity.com/paper/1909.13226/) (2019)
- [FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation](https://mlsanity.com/paper/1902.09513/) (2019)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [Track to Detect and Segment: An Online Multi-Object Tracker](https://mlsanity.com/paper/2103.08808/) (2021)
- [BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation](https://mlsanity.com/paper/2001.00309/) (2020)

## Built on by (most cited first, 13 of 13)

- [Personalize Segment Anything Model with One Shot](https://mlsanity.com/paper/2305.03048/) (2023)
- [In Defense of Online Models for Video Instance Segmentation](https://mlsanity.com/paper/2207.10661/) (2022)
- [MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Training](https://mlsanity.com/paper/2208.02245/) (2022)
- [CTVIS: Consistent Training for Online Video Instance Segmentation](https://mlsanity.com/paper/2307.12616/) (2023)
- [Prototypical Contrast Adaptation for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2207.06654/) (2022)
- [Rehabilitation Exercise Repetition Segmentation and Counting using Skeletal Body Joints](https://mlsanity.com/paper/2304.09735/) (2023)
- [Prompt-Driven Building Footprint Extraction in Aerial Images with Offset-Building Model](https://mlsanity.com/paper/2310.16717/) (2023)
- [A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects](https://mlsanity.com/paper/2506.13552/) (2025)
- [Dual Path Transformer with Partition Attention](https://mlsanity.com/paper/2305.14768/) (2023)
- [Local2Global query Alignment for Video Instance Segmentation](https://mlsanity.com/paper/2507.20120/) (2025)
- [LTCA: Long-range Temporal Context Attention for Referring Video Object Segmentation](https://mlsanity.com/paper/2510.08305/) (2025)
- [Structured prototype regularization for synthetic-to-real driving scene parsing](https://mlsanity.com/paper/2603.16083/) (2026)
- [SA-VIS: Sparse frame Annotations for training Video Instance Segmentation](https://mlsanity.com/paper/2606.20140/) (2026)

## Abstract (arXiv)

Video Instance Segmentation (VIS) is a task that simultaneously requires classification, segmentation, and instance association in a video. Recent VIS approaches rely on sophisticated pipelines to achieve this goal, including RoI-related operations or 3D convolutions. In contrast, we present a simple and efficient single-stage VIS framework based on the instance segmentation method CondInst by adding an extra tracking head. To improve instance association accuracy, a novel bi-directional spatio-temporal contrastive learning strategy for tracking embedding across frames is proposed. Moreover, an instance-wise temporal consistency scheme is utilized to produce temporally coherent results. Experiments conducted on the YouTube-VIS-2019, YouTube-VIS-2021, and OVIS-2021 datasets validate the effectiveness and efficiency of the proposed method. We hope the proposed framework can serve as a simple and strong alternative for many other instance-level video association tasks.