# CAVIS: Context-Aware Video Instance Segmentation

arXiv: https://arxiv.org/abs/2407.03010 · HTML: https://mlsanity.com/paper/2407.03010/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 65 references that are in the map.
- First posted on arXiv 3 Jul 2024.

## Builds on (most cited first, 30 of 51)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Vision Transformer Adapter for Dense Predictions](https://mlsanity.com/paper/2205.08534/) (2022)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation](https://mlsanity.com/paper/2106.05210/) (2021)
- [Mask2Former for Video Instance Segmentation](https://mlsanity.com/paper/2112.10764/) (2021)
- [Occluded Video Instance Segmentation: A Benchmark](https://mlsanity.com/paper/2102.01558/) (2021)
- [Video Panoptic Segmentation](https://mlsanity.com/paper/2006.11339/) (2020)
- [SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation](https://mlsanity.com/paper/2007.14772/) (2020)
- [Video Instance Segmentation using Inter-Frame Communication Transformers](https://mlsanity.com/paper/2106.03299/) (2021)
- [SeqFormer: Sequential Transformer for Video Instance Segmentation](https://mlsanity.com/paper/2112.08275/) (2021)
- [In Defense of Online Models for Video Instance Segmentation](https://mlsanity.com/paper/2207.10661/) (2022)
- [ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation](https://mlsanity.com/paper/2012.05258/) (2020)
- [VITA: Video Instance Segmentation via Object Token Association](https://mlsanity.com/paper/2206.04403/) (2022)

## Built on by (most cited first, 11 of 11)

- [VidEoMT: Your ViT is Secretly Also a Video Segmentation Model](https://mlsanity.com/paper/2602.17807/) (2026)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [Latest Object Memory Management for Temporally Consistent Video Instance Segmentation](https://mlsanity.com/paper/2507.19754/) (2025)
- [Evaluating SAM2 for Video Semantic Segmentation](https://mlsanity.com/paper/2512.01774/) (2025)
- [Beyond Appearance: Geometric Cues for Robust Video Instance Segmentation](https://mlsanity.com/paper/2507.05948/) (2025)
- [PMT: Plain Mask Transformer for Image and Video Segmentation with Frozen Vision Encoders](https://mlsanity.com/paper/2603.25398/) (2026)
- [A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects](https://mlsanity.com/paper/2506.13552/) (2025)
- [CRISP: Contrastive Residual Injection and Semantic Prompting for Continual Video Instance Segmentation](https://mlsanity.com/paper/2508.10432/) (2025)
- [AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes](https://mlsanity.com/paper/2606.02724/) (2026)
- [QueenVIS: Rethinking Image-Only Training for Video Instance Segmentation via Query Enrichment](https://mlsanity.com/paper/2607.24598/) (2026)
- [QCell: Recombining and Aligning Cell Queries for Overlapping Instance Segmentation](https://mlsanity.com/paper/2608.29253/) (2026)

## Abstract (arXiv)

In this paper, we introduce the Context-Aware Video Instance Segmentation (CAVIS), a novel framework designed to enhance instance association by integrating contextual information adjacent to each object. To efficiently extract and leverage this information, we propose the Context-Aware Instance Tracker (CAIT), which merges contextual data surrounding the instances with the core instance features to improve tracking accuracy. Additionally, we design the Prototypical Cross-frame Contrastive (PCC) loss, which ensures consistency in object-level features across frames, thereby significantly enhancing matching accuracy. CAVIS demonstrates superior performance over state-of-the-art methods on all benchmark datasets in video instance segmentation (VIS) and video panoptic segmentation (VPS). Notably, our method excels on the OVIS dataset, known for its particularly challenging videos. Project page: https://seung-hun-lee.github.io/projects/CAVIS/