mlsanityData to 7 Oct 2026

Video Instance Segmentation

arXiv 1905.04804 ↗First posted 12 May 2019cs.CVcited by 391

Not on the map. Why?

Abstract (arXiv)

In this paper we present a new computer vision task, named video instance segmentation. The goal of this new task is simultaneous detection, segmentation and tracking of instances in videos. In words, it is the first time that the image instance segmentation problem is extended to the video domain. To facilitate research on this new task, we propose a large-scale benchmark called YouTube-VIS, which consists of 2883 high-resolution YouTube videos, a 40-category label set and 131k high-quality instance masks. In addition, we propose a novel algorithm called MaskTrack R-CNN for this task. Our new method introduces a new tracking branch to Mask R-CNN to jointly perform the detection, segmentation and tracking tasks simultaneously. Finally, we evaluate the proposed method and several strong baselines on our new dataset. Experimental results clearly demonstrate the advantages of the proposed algorithm and reveal insight for future improvement. We believe the video instance segmentation task will motivate the community along the line of research for video understanding.

Key facts. Cited by 391 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 66; 2022: 70; 2023: 81; 2024: 67; 2025: 62; 2026: 44.

391 papers.

Sort:

By area:

  1. 2021Transformers in Vision: A Surveycited by 1,009
  2. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  3. 2023Segment Anything in High Qualitycited by 373
  4. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  5. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  6. 2021Associating Objects with Transformers for Video Object Segmentationcited by 225
  7. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  8. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  9. 2022Language as Queries for Referring Video Object Segmentationcited by 180
  10. 2021Mask2Former for Video Instance Segmentationcited by 169
  11. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  12. 2021Instances as Queriescited by 157
38 more
  1. 2021Track to Detect and Segment: An Online Multi-Object Trackercited by 156
  2. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  3. 2021Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networkscited by 141
  4. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  5. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  6. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  7. 2021Learning to Track with Object Permanencecited by 117
  8. 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
  9. 2021Video Instance Segmentation using Inter-Frame Communication Transformerscited by 105
  10. 2023SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentationcited by 103
  11. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  12. 2021SeqFormer: Sequential Transformer for Video Instance Segmentationcited by 98
  13. 2022In Defense of Online Models for Video Instance Segmentationcited by 97
  14. 2022VITA: Video Instance Segmentation via Object Token Associationcited by 94
  15. 2022MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Trainingcited by 94
  16. 2022Towards Grand Unification of Object Trackingcited by 93
  17. 2022High-Quality Entity Segmentationcited by 93
  18. 2021SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentationcited by 87
  19. 2021Crossover Learning for Fast Online Video Instance Segmentationcited by 83
  20. 2022Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentationcited by 79
  21. 2021Full-Duplex Strategy for Video Object Segmentationcited by 78
  22. 2022TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformerscited by 78
  23. 2023Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similaritiescited by 78
  24. 2023DVIS: Decoupled Video Instance Segmentation Frameworkcited by 77
  25. 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 76
  26. 2024Autoregressive Queries for Adaptive Tracking with Spatio-TemporalTransformerscited by 76
  27. 2021Video Instance Segmentation with a Propose-Reduce Paradigmcited by 74
  28. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 73
  29. 2021STEP: Segmenting and Tracking Every Pixelcited by 72
  30. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  31. 2022Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wildcited by 70
  32. 2021SwiftNet: Real-time Video Object Segmentationcited by 67
  33. 2022BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Videocited by 66
  34. 20214D Panoptic LiDAR Segmentationcited by 64
  35. 2021SiamMOT: Siamese Multi-Object Trackingcited by 62
  36. 2021Do Different Tracking Tasks Require Different Appearance Models?cited by 62
  37. 2022QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Trackingcited by 61
  38. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 61

50 most cited of 391 All 391 (JSON)