mlsanityData to 14 Oct 2026

Towards Unified Token Learning for Vision-Language Tracking

arXiv 2308.14103 ↗First posted 27 Aug 2023cs.CVcited by 62

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a simple, flexible and effective vision-language (VL) tracking pipeline, termed MMTrack, which casts VL tracking as a token generation task. Traditional paradigms address VL tracking task indirectly with sophisticated prior designs, making them over-specialize on the features of specific architectures or mechanisms. In contrast, our proposed framework serializes language description and bounding box into a sequence of discrete tokens. In this new design paradigm, all token queries are required to perceive the desired target and directly predict spatial coordinates of the target in an auto-regressive manner. The design without other prior modules avoids multiple sub-tasks learning and hand-designed loss functions, significantly reducing the complexity of VL tracking modeling and allowing our tracker to use a simple cross-entropy loss as unified optimization objective for VL tracking task. Extensive experiments on TNL2K, LaSOT, LaSOT_rmext and OTB99-Lang benchmarks show that our approach achieves promising results, compared to other state-of-the-arts.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Builds on 43 of its 56 references that are in the map. First posted on arXiv 27 Aug 2023.

Builds on

43 of 56 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2014Going Deeper with Convolutionscited by 4,241
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  9. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  10. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  11. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  12. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
31 more
  1. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  2. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 568
  3. 2018SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networkscited by 446
  4. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 429
  5. 2021Transformer Trackingcited by 412
  6. 2021Learning Spatio-Temporal Transformer for Visual Trackingcited by 350
  7. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 349
  8. 2019Learning Discriminative Model Prediction for Trackingcited by 344
  9. 2022Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Frameworkcited by 330
  10. 2022MixFormer: End-to-End Tracking with Iterative Mixed Attentioncited by 281
  11. 2020Ocean: Object-aware Anchor-free Trackingcited by 212
  12. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  13. 2020Siamese Box Adaptive Network for Visual Trackingcited by 189
  14. 2022AiATrack: Attention in Attention for Transformer Visual Trackingcited by 137
  15. 2020LaSOT: A High-quality Large-scale Single Object Tracking Benchmarkcited by 132
  16. 2021Learning Target Candidate Association to Keep Track of What Not to Trackcited by 131
  17. 2021Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmarkcited by 126
  18. 2021UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modelingcited by 122
  19. 2021Diverse Part Discovery: Occluded Person Re-identification with Part-Aware Transformercited by 120
  20. 2022Backbone is All Your Need: A Simplified Architecture for Visual Object Trackingcited by 119
  21. 2022Correlation-Aware Deep Trackingcited by 62
  22. 2023Joint Visual Grounding and Tracking with Natural Language Specificationcited by 59
  23. 2022Divert More Attention to Vision-Language Trackingcited by 47
  24. 2019Grounding-Tracking-Integrationcited by 32
  25. 2019Real-time Visual Object Tracking with Natural Language Descriptioncited by 26
  26. 2022Learning Target-aware Representation for Visual Tracking via Informative Interactionscited by 25
  27. 2018Describe and Attend to Track: Learning Natural Language guided Structural Representation and Visual Attention for Object Trackingcited by 20
  28. 2019Siamese Natural Language Tracker: Tracking by Natural Language Descriptions with Siamese Trackerscited by 13
  29. 2021SiamRCR: Reciprocal Classification and Regression for Visual Object Trackingcited by 9
  30. 2022Global Tracking via Ensemble of Local Trackerscited by 8
  31. 2022Beyond Greedy Search: Tracking by Multi-Agent Reinforcement Learning-based Beam Searchcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 9; 2025: 31; 2026: 19.

62 papers.

Sort:

By area:

  1. 2023iKUN: Speak to Trackers without Retrainingcited by 32
  2. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  3. 2025Towards Universal Modal Tracking with Online Dense Temporal Token Learningcited by 30
  4. 2025Variation-aware Vision Token Dropping for Faster Large Vision-Language Modelscited by 27
  5. 2025SwiftVLA: Unlocking Spatiotemporal Dynamics for Lightweight VLA Models at Minimal Overheadcited by 26
  6. 2024DTLLM-VLT: Diverse Text Generation for Visual Language Tracking Based on LLMcited by 24
  7. 2025Exploring Decoupled Spatio-Temporal Consistency Learning and Self-Prompting Evolution for Self-Supervised Trackingcited by 23
  8. 2024EchoTrack: Auditory Referring Multi-Object Tracking for Autonomous Drivingcited by 19
  9. 2024Efficient Fine-Tuning of Large Language Models for Automated Medical Documentationcited by 16
  10. 2025RefAV: Towards Planning-Centric Scenario Miningcited by 13
  11. 2025RobustGS: Unified Boosting of Feedforward 3D Gaussian Splatting under Low-Quality Conditionscited by 13
  12. 2024DTVLT: A Multi-modal Diverse Text Benchmark for Visual Language Tracking Based on LLMcited by 12
38 more
  1. 2023Integrating Language-Derived Appearance Elements with Visual Cues in Pedestrian Detectioncited by 10
  2. 2024Visual Language Tracking with Multi-modal Interaction: A Robust Benchmarkcited by 10
  3. 2024How Texts Help? A Fine-grained Evaluation to Reveal the Role of Language in Vision-Language Trackingcited by 10
  4. 2025ATCTrack: Aligning Target-Context Cues with Dynamic Target States for Robust Vision-Language Trackingcited by 10
  5. 2023Beyond Visual Cues: Synchronously Exploring Target-Centric Semantics for Vision-Language Trackingcited by 9
  6. 2025ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Trackingcited by 9
  7. 2024Enhancing Vision-Language Tracking by Effectively Converting Textual Cues into Visual Cuescited by 8
  8. 2024A GEN AI Framework for Medical Note Generationcited by 6
  9. 2025MEX: Memory-efficient Approach to Referring Multi-Object Trackingcited by 4
  10. 2025Beyond Frame-wise Tracking: A Trajectory-based Paradigm for Efficient Point Cloud Trackingcited by 4
  11. 2024Conditional Prototype Rectification Prompt Learningcited by 3
  12. 2025Diffusion-Based Image Editing: An Unforeseen Adversary to Robust Invisible Watermarkscited by 3
  13. 2025VPTracker: Global Vision-Language Tracking via Visual Promptcited by 3
  14. 2025COST: Contrastive One-Stage Transformer for Vision-Language Small Object Trackingcited by 2
  15. 2025CLDTracker: A Comprehensive Language Description for Visual Trackingcited by 2
  16. 2025ATSTrack: Enhancing Visual-Language Tracking by Aligning Temporal and Spatial Scalescited by 2
  17. 2025Omni Survey for Multimodality Analysis in Visual Object Trackingcited by 2
  18. 2025SSCM: A Spatial-Semantic Consistent Model for Multi-Contrast MRI Super-Resolutioncited by 2
  19. 2026Vanishing Watermarks: Diffusion-Based Image Editing Undermines Robust Invisible Watermarkingcited by 2
  20. 2026RAGTrack: Language-aware RGBT Tracking with Retrieval-Augmented Generationcited by 2
  21. 2026Hypergraph-State Collaborative Reasoning for Multi-Object Trackingcited by 2
  22. 20264DVLT: Dynamic Scene Understanding with Worldline-Centered Vision-Language Trackingcited by 2
  23. 2025Towards General Multimodal Visual Trackingcited by 1
  24. 2025Geometrically Regularized Transfer Learning with On-Manifold and Off-Manifold Perturbationcited by 1
  25. 2025Improving Token-based Object Detection with Videocited by 1
  26. 2025Learning Frequency and Memory-Aware Prompts for Multi-Modal Object Trackingcited by 1
  27. 2025Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reportscited by 1
  28. 2025From Seeing to Predicting: A Vision-Language Framework for Trajectory Forecasting and Controlled Video Generationcited by 1
  29. 2026GLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templatescited by 1
  30. 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offscited by 1
  31. 2026GS-STVSR: Ultra-Efficient Continuous Spatio-Temporal Video Super-Resolution via 2D Gaussian Splattingcited by 1
  32. 2026Boosting Self-Supervised Tracking with Contextual Prompts and Noise Learningcited by 1
  33. 2026Learning to Track Instance from Single Nature Language Descriptioncited by 1
  34. 2026An Efficient Token Compression Framework for Visual Object Trackingcited by 1
  35. 2025Disentangled Geometric Alignment with Adaptive Contrastive Perturbation for Reliable Domain Transfercited by 0
  36. 2025Dynamic Epsilon Scheduling: A Multi-Factor Adaptive Perturbation Budget for Adversarial Trainingcited by 0
  37. 2025FADE: Adversarial Concept Erasure in Flow Modelscited by 0
  38. 2025SOI is the Root of All Evil: Quantifying and Breaking Similar Object Interference in Single Object Trackingcited by 0

50 most cited of 62 All 62 (JSON)