mlsanityData to 14 Oct 2026

OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking

arXiv 2410.17534 ↗First posted 23 Oct 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Open-vocabulary object perception has become an important topic in artificial intelligence, which aims to identify objects with novel classes that have not been seen during training. Under this setting, open-vocabulary object detection (OVD) in a single image has been studied in many literature. However, open-vocabulary object tracking (OVT) from a video has been studied less, and one reason is the shortage of benchmarks. In this work, we have built a new large-scale benchmark for open-vocabulary multi-object tracking namely OVT-B. OVT-B contains 1,048 categories of objects and 1,973 videos with 637,608 bounding box annotations, which is much larger than the sole open-vocabulary tracking dataset, i.e., OVTAO-val dataset (200+ categories, 900+ videos). The proposed OVT-B can be used as a new benchmark to pave the way for OVT research. We also develop a simple yet effective baseline method for OVT. It integrates the motion features for object tracking, which is an important feature for MOT but is ignored in previous OVT methods. Experimental results have verified the usefulness of the proposed benchmark and the effectiveness of our method. We have released the benchmark to the public at https://github.com/Coo1Sea/OVT-B-Dataset.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 53 of its 66 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

53 of 66 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  8. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  9. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
  10. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  11. 2016Simple Online and Realtime Trackingcited by 868
  12. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
38 more
  1. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  2. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 532
  3. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 518
  4. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 490
  5. 2020Tracking Objects as Pointscited by 483
  6. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  7. 2019Tracking without bells and whistlescited by 382
  8. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 381
  9. 2020MOT20: A benchmark for multi object tracking in crowded scenescited by 357
  10. 2022Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Trackingcited by 346
  11. 2022BoT-SORT: Robust Associations Multi-Pedestrian Trackingcited by 320
  12. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  13. 2019Towards Real-Time Multi-Object Trackingcited by 311
  14. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  15. 2021DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motioncited by 217
  16. 2022StrongSORT: Make DeepSORT Great Againcited by 213
  17. 2018The Unmanned Aerial Vehicle Benchmark: Object Detection and Trackingcited by 208
  18. 2019Learning a Neural Solver for Multiple Object Trackingcited by 198
  19. 2023Towards Open Vocabulary Learning: A Surveycited by 166
  20. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  21. 2020TAO: A Large-Scale Benchmark for Tracking Any Objectcited by 147
  22. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  23. 2022PromptDet: Towards Open-vocabulary Detection using Uncurated Imagescited by 133
  24. 2023Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identificationcited by 132
  25. 2017Detect to Track and Track to Detectcited by 131
  26. 2023Aligning Bag of Regions for Open-Vocabulary Object Detectioncited by 119
  27. 2022Exploiting Unlabeled Data with Vision and Language Models for Object Detectioncited by 97
  28. 2022Learning Object-Language Alignments for Open-Vocabulary Object Detectioncited by 96
  29. 2016POI: Multiple Object Tracking with High Performance Detection and Appearance Featurecited by 74
  30. 2022QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Trackingcited by 61
  31. 2023OVTrack: Open-Vocabulary Multiple Object Trackingcited by 53
  32. 2021Opening up Open-World Trackingcited by 50
  33. 2023CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detectioncited by 46
  34. 2023Towards Open-Vocabulary Video Instance Segmentationcited by 45
  35. 2022Tracking Every Thing in the Wildcited by 44
  36. 2022AnimalTrack: A Benchmark for Multi-Animal Tracking in the Wildcited by 37
  37. 2020GMOT-40: A Benchmark for Generic Multiple Object Trackingcited by 32
  38. 2023Contrastive Feature Masking Open-Vocabulary Vision Transformercited by 31

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025OVTR: End-to-End Open-Vocabulary Multiple Object Tracking with Transformercited by 11
  2. 2025Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Surveycited by 2
  3. 2026TSBOW -- Traffic Surveillance Benchmark for Occluded Vehicles Under Various Weather Conditionscited by 2
  4. 2025Lattice Boltzmann Model for Learning Real-World Pixel Dynamicitycited by 0
  5. 2026SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoningcited by 0