mlsanityData to 7 Oct 2026

Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentation

arXiv 2104.04691 ↗First posted 10 Apr 2021cs.CVcited by 133

Not on the map. Why?

Abstract (arXiv)

Current state-of-the-art object detection and segmentation methods work well under the closed-world assumption. This closed-world setting assumes that the list of object categories is available during training and deployment. However, many real-world applications require detecting or segmenting novel objects, i.e., object categories never seen during training. In this paper, we present, UVO (Unidentified Video Objects), a new benchmark for open-world class-agnostic object segmentation in videos. Besides shifting the problem focus to the open-world setup, UVO is significantly larger, providing approximately 8 times more videos compared with DAVIS, and 7 times more mask (instance) annotations per video compared with YouTube-VOS and YouTube-VIS. UVO is also more challenging as it includes many videos with crowded scenes and complex background motions. We demonstrated that UVO can be used for other applications, such as object tracking and super-voxel segmentation, besides open-world object segmentation. We believe that UVo is a versatile testbed for researchers to develop novel approaches for open-world class-agnostic object segmentation, and inspires new research directions towards a more comprehensive video understanding beyond classification and detection.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). Builds on 30 of its 55 references that are in the map. First posted on arXiv 10 Apr 2021.

Builds on

30 of 55 references.

References · 30

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2017Mask R-CNNcited by 7,259
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2017Non-local Neural Networkscited by 2,290
  7. 2017The Kinetics Human Action Video Datasetcited by 1,894
  8. 2018SlowFast Networks for Video Recognitioncited by 1,862
  9. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  10. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  11. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  12. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
18 more
  1. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  2. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  3. 2019Video Instance Segmentationcited by 391
  4. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
  5. 2019Video Classification with Channel-Separated Convolutional Networkscited by 290
  6. 2014Towards Open World Recognitioncited by 226
  7. 2019MOTS: Multi-Object Tracking and Segmentationcited by 222
  8. 2018Videos as Space-Time Region Graphscited by 221
  9. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  10. 2020TAO: A Large-Scale Benchmark for Tracking Any Objectcited by 146
  11. 2020Don't Judge an Object by Its Context: Learning to Overcome Contextual Biascited by 91
  12. 2015Learning to Segment Object Candidatescited by 79
  13. 2017Learning to Segment Every Thingcited by 64
  14. 2020Class-agnostic Object Detectioncited by 17
  15. 2020All About Knowledge Graphs for Actionscited by 16
  16. 2018Pixel Objectness: Learning to Segment Generic Objects Automatically in Images and Videoscited by 9
  17. 2017End-to-End Video Classification with Knowledge Graphscited by 3
  18. 2015LIBSVX: A Supervoxel Library and Benchmark for Early Video Processingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 19; 2023: 39; 2024: 31; 2025: 24; 2026: 10.

133 papers.

Sort:

By area:

  1. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  2. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  3. 2023Segment Anything in High Qualitycited by 373
  4. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  5. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  6. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  7. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  8. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 186
  9. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  10. 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 139
  11. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  12. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
38 more
  1. 2024VISA: Reasoning Video Object Segmentation via Large Language Modelscited by 123
  2. 2022Global Tracking Transformerscited by 110
  3. 2021Learning Open-World Object Proposals without Learning to Classifycited by 109
  4. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  5. 2022FreeSOLO: Learning to Segment Objects without Annotationscited by 102
  6. 2022High-Quality Entity Segmentationcited by 93
  7. 2021Class-agnostic Object Detection with Multi-modal Transformercited by 85
  8. 2023Segment Anything Meets Point Trackingcited by 81
  9. 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 76
  10. 2021Open-World Entity Segmentationcited by 74
  11. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 73
  12. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  13. 2022BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Videocited by 66
  14. 2021Opening up Open-World Trackingcited by 50
  15. 2024Matching Anything by Segmenting Anythingcited by 49
  16. 2022Breaking the "Object" in Video Object Segmentationcited by 48
  17. 2023Towards Open-Vocabulary Video Instance Segmentationcited by 45
  18. 2022Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinitycited by 41
  19. 2024LVOS: A Benchmark for Large-scale Long-term Video Object Segmentationcited by 41
  20. 2023Open-vocabulary Panoptic Segmentation with Embedding Modulationcited by 39
  21. 2024Mind the Time: Temporally-Controlled Multi-Event Video Generationcited by 39
  22. 2021Learning to Detect Every Thing in an Open Worldcited by 37
  23. 2024LW-DETR: A Transformer Replacement to YOLO for Real-Time Detectioncited by 36
  24. 2023Unsupervised Universal Image Segmentationcited by 34
  25. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 34
  26. 2022Revisiting Open World Object Detectioncited by 30
  27. 2023EgoTracks: A Long-term Egocentric Visual Object Tracking Datasetcited by 27
  28. 2023StoryBench: A Multifaceted Benchmark for Continuous Story Visualizationcited by 27
  29. 2024Segment anything model 2: an application to 2D and 3D medical imagescited by 27
  30. 2023OpenVIS: Open-vocabulary Video Instance Segmentationcited by 25
  31. 2022Is an Object-Centric Video Representation Beneficial for Transfer?cited by 24
  32. 2023Connecting Vision and Language with Video Localized Narrativescited by 24
  33. 2022OpenTAL: Towards Open Set Temporal Action Localizationcited by 23
  34. 2025EdgeTAM: On-Device Track Anything Modelcited by 21
  35. 2024Distilling Vision-Language Models on Millions of Videoscited by 20
  36. 2023Cascade-DETR: Delving into High-Quality Universal Object Detectioncited by 19
  37. 2023Learning the What and How of Annotation in Video Object Segmentationcited by 19
  38. 2025EgoExo-Gen: Ego-centric Video Prediction by Watching Exo-centric Videoscited by 19

50 most cited of 133 All 133 (JSON)