mlsanityData to 7 Oct 2026

CAVIS: Context-Aware Video Instance Segmentation

arXiv 2407.03010 ↗First posted 3 Jul 2024cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce the Context-Aware Video Instance Segmentation (CAVIS), a novel framework designed to enhance instance association by integrating contextual information adjacent to each object. To efficiently extract and leverage this information, we propose the Context-Aware Instance Tracker (CAIT), which merges contextual data surrounding the instances with the core instance features to improve tracking accuracy. Additionally, we design the Prototypical Cross-frame Contrastive (PCC) loss, which ensures consistency in object-level features across frames, thereby significantly enhancing matching accuracy. CAVIS demonstrates superior performance over state-of-the-art methods on all benchmark datasets in video instance segmentation (VIS) and video panoptic segmentation (VPS). Notably, our method excels on the OVIS dataset, known for its particularly challenging videos. Project page: https://seung-hun-lee.github.io/projects/CAVIS/

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 51 of its 65 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

51 of 65 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2017Mask R-CNNcited by 7,259
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  10. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  11. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  12. 2022Vision Transformer Adapter for Dense Predictionscited by 564
38 more
  1. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 531
  2. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 489
  3. 2020Tracking Objects as Pointscited by 483
  4. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  5. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  6. 2019Video Instance Segmentationcited by 391
  7. 2019Tracking without bells and whistlescited by 382
  8. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
  9. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 200
  10. 2021Mask2Former for Video Instance Segmentationcited by 169
  11. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  12. 2020Video Panoptic Segmentationcited by 141
  13. 2020SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentationcited by 108
  14. 2021Video Instance Segmentation using Inter-Frame Communication Transformerscited by 105
  15. 2021SeqFormer: Sequential Transformer for Video Instance Segmentationcited by 98
  16. 2022In Defense of Online Models for Video Instance Segmentationcited by 97
  17. 2020ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentationcited by 95
  18. 2022VITA: Video Instance Segmentation via Object Token Associationcited by 94
  19. 2022MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Trainingcited by 94
  20. 2021Crossover Learning for Fast Online Video Instance Segmentationcited by 83
  21. 2022Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentationcited by 79
  22. 2023DVIS: Decoupled Video Instance Segmentation Frameworkcited by 77
  23. 2019Spatial-Temporal Relation Networks for Multi-Object Trackingcited by 74
  24. 2022Temporally Efficient Vision Transformer for Video Instance Segmentationcited by 55
  25. 2022A Generalized Framework for Video Instance Segmentationcited by 52
  26. 2023CTVIS: Consistent Training for Online Video Instance Segmentationcited by 50
  27. 2023DVIS++: Improved Decoupled Framework for Universal Video Segmentationcited by 46
  28. 2018Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Trackingcited by 40
  29. 2022Efficient Video Instance Segmentation via Tracklet Query and Proposalcited by 33
  30. 2023TarViS: A Unified Approach for Target-based Video Segmentationcited by 32
  31. 2021VISOLO: Grid-Based Space-Time Aggregation for Efficient Online Video Instance Segmentationcited by 29
  32. 2021Learning to Associate Every Segment for Video Panoptic Segmentationcited by 25
  33. 2023Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentationcited by 24
  34. 2023Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentationcited by 23
  35. 2023TCOVIS: Temporally Consistent Online Video Instance Segmentationcited by 18
  36. 2022BATMAN: Bilateral Attention Transformer in Motion-Appearance Neighboring Space for Video Object Segmentationcited by 14
  37. 2023MDQE: Mining Discriminative Query Embeddings to Segment Occluded Instances on Challenging Videoscited by 13
  38. 2023VISAGE: Video Instance Segmentation with Appearance-Guided Enhancementcited by 7

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2026VidEoMT: Your ViT is Secretly Also a Video Segmentation Modelcited by 7
  2. 2026Video Understanding: From Geometry and Semantics to Unified Modelscited by 5
  3. 2025Latest Object Memory Management for Temporally Consistent Video Instance Segmentationcited by 4
  4. 2025Evaluating SAM2 for Video Semantic Segmentationcited by 4
  5. 2025Beyond Appearance: Geometric Cues for Robust Video Instance Segmentationcited by 3
  6. 2026PMT: Plain Mask Transformer for Image and Video Segmentation with Frozen Vision Encoderscited by 3
  7. 2025A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospectscited by 2
  8. 2025CRISP: Contrastive Residual Injection and Semantic Prompting for Continual Video Instance Segmentationcited by 2
  9. 2026AVTrack: Audio-Visual Tracking in Human-centric Complex Scenescited by 0
  10. 2026QueenVIS: Rethinking Image-Only Training for Video Instance Segmentation via Query Enrichmentcited by 0
  11. 2026QCell: Recombining and Aligning Cell Queries for Overlapping Instance Segmentationcited by 0