mlsanityData to 7 Oct 2026

Occluded Video Instance Segmentation: A Benchmark

arXiv 2102.01558 ↗First posted 2 Feb 2021cs.CVcited by 154

Not on the map. Why?

Abstract (arXiv)

Can our video understanding systems perceive objects when a heavy occlusion exists in a scene? To answer this question, we collect a large-scale dataset called OVIS for occluded video instance segmentation, that is, to simultaneously detect, segment, and track instances in occluded scenes. OVIS consists of 296k high-quality instance masks from 25 semantic categories, where object occlusions usually occur. While our human vision systems can understand those occluded instances by contextual reasoning and association, our experiments suggest that current video understanding systems cannot. On the OVIS dataset, the highest AP achieved by state-of-the-art algorithms is only 16.3, which reveals that we are still at a nascent stage for understanding objects, instances, and videos in a real-world scenario. We also present a simple plug-and-play module that performs temporal feature calibration to complement missing object cues caused by occlusion. Built upon MaskTrack R-CNN and SipMask, we obtain a remarkable AP improvement on the OVIS dataset. The OVIS dataset and project code are available at http://songbai.site/ovis .

Key facts. Cited by 154 papers in the map (data through 7 Oct 2026). Builds on 75 of its 104 references that are in the map. First posted on arXiv 2 Feb 2021.

Builds on

75 of 104 references.

References · 75

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2017Mask R-CNNcited by 7,259
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  9. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  10. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  11. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  12. 2017Non-local Neural Networkscited by 2,290
38 more
  1. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  2. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  3. 2017Deformable Convolutional Networkscited by 1,481
  4. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  5. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  6. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 908
  7. 2018Panoptic Segmentationcited by 796
  8. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  9. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 531
  10. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  11. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  12. 2019PointRend: Image Segmentation as Renderingcited by 439
  13. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  14. 2019Video Instance Segmentationcited by 391
  15. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
  16. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  17. 2018CrowdHuman: A Benchmark for Detecting Human in a Crowdcited by 283
  18. 2019Mask Scoring R-CNNcited by 265
  19. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 228
  20. 2019MOTS: Multi-Object Tracking and Segmentationcited by 222
  21. 2017Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detectioncited by 209
  22. 2016Deep Feature Flow for Video Recognitioncited by 186
  23. 2019FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentationcited by 178
  24. 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
  25. 2017CityPersons: A Diverse Dataset for Pedestrian Detectioncited by 171
  26. 2016Learning Video Object Segmentation from Static Imagescited by 151
  27. 2020Video Panoptic Segmentationcited by 141
  28. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  29. 2019Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagationcited by 125
  30. 2020STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videoscited by 121
  31. 2019Synthetic Data for Deep Learningcited by 113
  32. 2020TIDE: A General Toolbox for Identifying Object Detection Errorscited by 113
  33. 2017Learning non-maximum suppressioncited by 112
  34. 2017Online Adaptation of Convolutional Neural Networks for Video Object Segmentationcited by 109
  35. 2020SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentationcited by 108
  36. 2015UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Trackingcited by 98
  37. 2018Efficient Video Object Segmentation via Network Modulationcited by 98
  38. 2020Self-Supervised Scene De-occlusioncited by 96

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 13; 2023: 37; 2024: 40; 2025: 34; 2026: 29.

154 papers.

Sort:

By area:

  1. 2023Segment Anythingcited by 8,158
  2. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  3. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  4. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  5. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  6. 2023Putting the Object Back into Video Object Segmentationcited by 182
  7. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 161
  8. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  9. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  10. 2021SeqFormer: Sequential Transformer for Video Instance Segmentationcited by 98
  11. 2022In Defense of Online Models for Video Instance Segmentationcited by 97
  12. 2022MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Trainingcited by 94
38 more
  1. 2023DVIS: Decoupled Video Instance Segmentation Frameworkcited by 77
  2. 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 76
  3. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  4. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 59
  5. 2023CTVIS: Consistent Training for Online Video Instance Segmentationcited by 50
  6. 2023Advances in Deep Concealed Scene Understandingcited by 47
  7. 2023Towards Open-Vocabulary Video Instance Segmentationcited by 45
  8. 2024LVOS: A Benchmark for Large-scale Long-term Video Object Segmentationcited by 41
  9. 2025DICEPTION: A Generalist Diffusion Model for Visual Perceptual Taskscited by 37
  10. 2025MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligencecited by 34
  11. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 34
  12. 2023TarViS: A Unified Approach for Target-based Video Segmentationcited by 32
  13. 2023Connecting Vision and Language with Video Localized Narrativescited by 24
  14. 2023Type-to-Track: Retrieve Any Object via Prompt-based Trackingcited by 24
  15. 2023Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentationcited by 23
  16. 2023UniRef++: Segment Every Reference Object in Spatial and Temporal Spacescited by 22
  17. 2022A Tri-Layer Plugin to Improve Occluded Detectioncited by 21
  18. 2023Audio-Visual Instance Segmentationcited by 21
  19. 2023Mask-Free Video Instance Segmentationcited by 20
  20. 2024Distilling Vision-Language Models on Millions of Videoscited by 20
  21. 2023Learning the What and How of Annotation in Video Object Segmentationcited by 19
  22. 2022Region Aware Video Object Segmentation with Deep Motion Modelingcited by 18
  23. 2022YOLOV: Making Still Image Object Detectors Great at Video Object Detectioncited by 18
  24. 2023TCOVIS: Temporally Consistent Online Video Instance Segmentationcited by 18
  25. 2025Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Markscited by 17
  26. 2024ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded Segmentationcited by 16
  27. 2025Are They the Same? Exploring Visual Correspondence Shortcomings of Multimodal LLMscited by 16
  28. 2024Unified Embedding Alignment for Open-Vocabulary Video Instance Segmentationcited by 15
  29. 2024PartGLEE: A Foundation Model for Recognizing and Parsing Any Objectscited by 15
  30. 2024ZIM: Zero-Shot Image Matting for Anythingcited by 15
  31. 2023UVOSAM: A Mask-free Paradigm for Unsupervised Video Object Segmentation via Segment Anything Modelcited by 14
  32. 2025LanP: Rethinking the Impact of Language Priors in Large Vision-Language Modelscited by 14
  33. 2023InsPro: Propagating Instance Query and Proposal for Online Video Instance Segmentationcited by 13
  34. 2024CLIP-VIS: Adapting CLIP for Open-Vocabulary Video Instance Segmentationcited by 13
  35. 2025SAM2 for Image and Video Segmentation: A Comprehensive Surveycited by 13
  36. 2025Reasoning Segmentation for Images and Videos: A Surveycited by 12
  37. 2025Object-centric Video Question Answering with Visual Grounding and Referringcited by 12
  38. 2024SAM-PM: Enhancing Video Camouflaged Object Detection using Spatio-Temporal Attentioncited by 11

50 most cited of 154 All 154 (JSON)