# VOVTrack: Exploring the Potentiality in Videos for Open-Vocabulary Object Tracking

arXiv: https://arxiv.org/abs/2410.08529 · HTML: https://mlsanity.com/paper/2410.08529/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 60 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Builds on (most cited first, 6 of 6)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [1st Place Solution to ECCV-TAO-2020: Detect and Represent Any Object for Tracking](https://mlsanity.com/paper/2101.08040/) (2021)
- [Pedestrian Tracking with Monocular Camera using Unconstrained 3D Motion Model](https://mlsanity.com/paper/2403.11978/) (2024)

## Built on by (most cited first, 4 of 4)

- [Open-World Object Counting in Videos](https://mlsanity.com/paper/2506.15368/) (2025)
- [OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking](https://mlsanity.com/paper/2410.17534/) (2024)
- [Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2510.10671/) (2025)
- [NOVA: Next-step Open-Vocabulary Autoregression for 3D Multi-Object Tracking in Autonomous Driving](https://mlsanity.com/paper/2603.06254/) (2026)

## Abstract (arXiv)

Open-vocabulary multi-object tracking (OVMOT) represents a critical new challenge involving the detection and tracking of diverse object categories in videos, encompassing both seen categories (base classes) and unseen categories (novel classes). This issue amalgamates the complexities of open-vocabulary object detection (OVD) and multi-object tracking (MOT). Existing approaches to OVMOT often merge OVD and MOT methodologies as separate modules, predominantly focusing on the problem through an image-centric lens. In this paper, we propose VOVTrack, a novel method that integrates object states relevant to MOT and video-centric training to address this challenge from a video object tracking standpoint. First, we consider the tracking-related state of the objects during tracking and propose a new prompt-guided attention mechanism for more accurate localization and classification (detection) of the time-varying objects. Subsequently, we leverage raw video data without annotations for training by formulating a self-supervised object similarity learning technique to facilitate temporal object association (tracking). Experimental results underscore that VOVTrack outperforms existing methods, establishing itself as a state-of-the-art solution for open-vocabulary tracking task.