mlsanityData to 13 Oct 2026

Mask2Former for Video Instance Segmentation

arXiv 2112.10764 ↗First posted 20 Dec 2021cs.CV cs.AI cs.LGcited by 170

Not on the map. Why?

Abstract (arXiv)

We find Mask2Former also achieves state-of-the-art performance on video instance segmentation without modifying the architecture, the loss or even the training pipeline. In this report, we show universal image segmentation architectures trivially generalize to video segmentation by directly predicting 3D segmentation volumes. Specifically, Mask2Former sets a new state-of-the-art of 60.4 AP on YouTubeVIS-2019 and 52.6 AP on YouTubeVIS-2021. We believe Mask2Former is also capable of handling video semantic and panoptic segmentation, given its versatility in image segmentation. We hope this will make state-of-the-art video segmentation research more accessible and bring more attention to designing universal image and video segmentation architectures.

Key facts. Cited by 170 papers in the map (data through 13 Oct 2026). Builds on 15 of its 17 references that are in the map. First posted on arXiv 20 Dec 2021.

Builds on

15 of 17 references.

References · 15

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,111
  4. 2020End-to-End Object Detection with Transformerscited by 7,396
  5. 2017Mask R-CNNcited by 7,266
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,344
  7. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,150
  8. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  9. 2019Video Instance Segmentationcited by 393
  10. 2019Panoptic-DeepLabcited by 362
  11. 2019MOTS: Multi-Object Tracking and Segmentationcited by 222
  12. 2020Video Panoptic Segmentationcited by 142
3 more
  1. 2020STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videoscited by 121
  2. 2021Video Instance Segmentation using Inter-Frame Communication Transformerscited by 105
  3. 2020ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentationcited by 96

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 23; 2023: 65; 2024: 37; 2025: 30; 2026: 15.

170 papers.

Sort:

By area:

  1. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 919
  2. 2023Segment Anything in High Qualitycited by 374
  3. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  4. 2022DETRs with Hybrid Matchingcited by 150
  5. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  6. 2024VISA: Reasoning Video Object Segmentation via Large Language Modelscited by 123
  7. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  8. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  9. 2022VITA: Video Instance Segmentation via Object Token Associationcited by 94
  10. 2022MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Trainingcited by 94
  11. 2022High-Quality Entity Segmentationcited by 93
  12. 2025OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generationcited by 83
38 more
  1. 2023Segment Anything Meets Point Trackingcited by 81
  2. 2023DVIS: Decoupled Video Instance Segmentation Frameworkcited by 77
  3. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 74
  4. 2022BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Videocited by 66
  5. 2023Learning Mask-aware CLIP Representations for Zero-Shot Segmentationcited by 57
  6. 2024MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentationcited by 56
  7. 2023Contrastive Grouping with Transformer for Referring Image Segmentationcited by 54
  8. 2022A Generalized Framework for Video Instance Segmentationcited by 52
  9. 2023CTVIS: Consistent Training for Online Video Instance Segmentationcited by 50
  10. 2022Connecting the Dots: Floorplan Reconstruction Using Two-Level Queriescited by 48
  11. 2023Spectrum-guided Multi-granularity Referring Video Object Segmentationcited by 48
  12. 2023DVIS++: Improved Decoupled Framework for Universal Video Segmentationcited by 46
  13. 2023Towards Open-Vocabulary Video Instance Segmentationcited by 45
  14. 2023Detection Transformer with Stable Matchingcited by 44
  15. 2023Global Knowledge Calibration for Fast Open-Vocabulary Segmentationcited by 40
  16. 2023M2T2: Multi-Task Masked Transformer for Object-centric Pick and Placecited by 40
  17. 2023VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentationcited by 37
  18. 2024Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoderscited by 36
  19. 2023LipsFormer: Introducing Lipschitz Continuity to Vision Transformerscited by 34
  20. 2022Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuningcited by 33
  21. 2023TarViS: A Unified Approach for Target-based Video Segmentationcited by 32
  22. 2024Ref-AVS: Refer and Segment Objects in Audio-Visual Scenescited by 32
  23. 2023Helping Hands: An Object-Aware Ego-Centric Video Recognition Modelcited by 31
  24. 2023Neural Implicit Dense Semantic SLAMcited by 26
  25. 2024UniVS: Unified and Universal Video Segmentation with Prompts as Queriescited by 26
  26. 2023OpenVIS: Open-vocabulary Video Instance Segmentationcited by 25
  27. 2022DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Groundingcited by 24
  28. 2023Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentationcited by 24
  29. 2023Mask4Former: Mask Transformer for 4D Panoptic Segmentationcited by 24
  30. 2023Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentationcited by 23
  31. 2024ViLLa: Video Reasoning Segmentation with Large Language Modelcited by 23
  32. 2024DVIS-DAQ: Improving Video Segmentation via Dynamic Anchor Queriescited by 22
  33. 2025Primus: Enforcing Attention Usage for 3D Medical Image Segmentationcited by 22
  34. 2023Audio-Visual Instance Segmentationcited by 21
  35. 2024GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervisioncited by 21
  36. 2022VMFormer: End-to-End Video Matting with Transformercited by 20
  37. 2023Mask-Free Video Instance Segmentationcited by 20
  38. 2024kNN-CLIP: Retrieval Enables Training-Free Segmentation on Continually Expanding Large Vocabulariescited by 20

50 most cited of 170 All 170 (JSON)