TP-GMOT: Tracking Generic Multiple Object by Textual Prompt with Motion-Appearance Cost (MAC) SORT

While Multi-Object Tracking (MOT) has made substantial advancements, it is limited by heavy reliance on prior knowledge and limited to predefined categories. In contrast, Generic Multiple Object Tracking (GMOT), tracking multiple objects with similar appearance, requires less prior information about the targets but faces challenges with variants like viewpoint, lighting, occlusion, and resolution. Our contributions commence with the introduction of the textbfRefer-GMOT dataset a collection of videos, each accompanied by fine-grained textual descriptions of their attributes. Subsequently, we introduce a novel text prompt-based open-vocabulary GMOT framework, called textbfTP-GMOT, which can track never-seen object categories with zero training examples. Within TP-GMOT framework, we introduce two novel components: (i) textbfTP-OD, an object detection by a textual prompt, for accurately detecting unseen objects with specific characteristics. (ii) Motion-Appearance Cost SORT textbfMAC-SORT, a novel object association approach that adeptly integrates motion and appearance-based matching strategies to tackle the complex task of tracking multiple generic objects with high similarity. Our contributions are benchmarked on the Refer-GMOT dataset for GMOT task. Additionally, to assess the generalizability of the proposed TP-GMOT framework and the effectiveness of MAC-SORT tracker, we conduct ablation studies on the DanceTrack and MOT20 datasets for the MOT task. Our dataset, code, and models will be publicly available at: https://fsoft-aic.github.io/TP-GMOT