mlsanityData to 14 Oct 2026

OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning

arXiv 2403.09634 ↗First posted 14 Mar 2024cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

Visual object tracking aims to localize the target object of each frame based on its initial appearance in the first frame. Depending on the input modility, tracking tasks can be divided into RGB tracking and RGB+X (e.g. RGB+N, and RGB+D) tracking. Despite the different input modalities, the core aspect of tracking is the temporal matching. Based on this common ground, we present a general framework to unify various tracking tasks, termed as OneTracker. OneTracker first performs a large-scale pre-training on a RGB tracker called Foundation Tracker. This pretraining phase equips the Foundation Tracker with a stable ability to estimate the location of the target object. Then we regard other modality information as prompt and build Prompt Tracker upon Foundation Tracker. Through freezing the Foundation Tracker and only adjusting some additional trainable parameters, Prompt Tracker inhibits the strong localization ability from Foundation Tracker and achieves parameter-efficient finetuning on downstream RGB+X tracking tasks. To evaluate the effectiveness of our general framework OneTracker, which is consisted of Foundation Tracker and Prompt Tracker, we conduct extensive experiments on 6 popular tracking tasks across 11 benchmarks and our OneTracker outperforms other models and achieves state-of-the-art performance.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Builds on 84 of its 118 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

84 of 118 references.

References · 84

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  12. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
38 more
  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  2. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  3. 2022Visual Prompt Tuningcited by 1,681
  4. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  5. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  6. 2021Scaling Vision Transformerscited by 999
  7. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  8. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  9. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  10. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  11. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  12. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  13. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 568
  14. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
  15. 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 475
  16. 2018SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networkscited by 446
  17. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
  18. 2019Video Object Segmentation using Space-Time Memory Networkscited by 438
  19. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 431
  20. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 429
  21. 2021Transformer Trackingcited by 412
  22. 2021Learning Spatio-Temporal Transformer for Visual Trackingcited by 350
  23. 2019Learning Discriminative Model Prediction for Trackingcited by 344
  24. 2018TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wildcited by 343
  25. 2022Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Frameworkcited by 330
  26. 2018ATOM: Accurate Tracking by Overlap Maximizationcited by 285
  27. 2022MixFormer: End-to-End Tracking with Iterative Mixed Attentioncited by 281
  28. 2018Fast Online Object Tracking and Segmentation: A Unifying Approachcited by 272
  29. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
  30. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 232
  31. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 231
  32. 2020Probabilistic Regression for Visual Trackingcited by 231
  33. 2021Associating Objects with Transformers for Video Object Segmentationcited by 226
  34. 2015Learning Multi-Domain Convolutional Neural Networks for Visual Trackingcited by 209
  35. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 192
  36. 2020Siamese Box Adaptive Network for Visual Trackingcited by 189
  37. 2019Siam R-CNN: Visual Tracking by Re-Detectioncited by 174
  38. 2020BoxInst: High-Performance Instance Segmentation with Box Annotationscited by 151

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 14; 2025: 29; 2026: 22.

65 papers.

Sort:

By area:

  1. 2024Exploiting Multimodal Spatial-temporal Patterns for Video Object Trackingcited by 57
  2. 2024XTrack: Multimodal Training Boosts RGB-X Video Object Trackerscited by 21
  3. 2025Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Maskingcited by 21
  4. 2024RGBT Tracking via All-layer Multimodal Interactions with Progressive Fusion Mambacited by 17
  5. 2025What You Have is What You Track: Adaptive and Robust Multimodal Trackingcited by 15
  6. 2024Cross-modulated Attention Transformer for RGBT Trackingcited by 14
  7. 2024MambaVT: Spatio-Temporal Contextual Modeling for robust RGB-T Trackingcited by 14
  8. 2024Visible-Thermal Multiple Object Tracking: Large-scale Video Dataset and Progressive Fusion Approachcited by 13
  9. 2024Breaking Modality Gap in RGBT Tracking: Coupled Knowledge Distillationcited by 11
  10. 2025Adaptive Perception for Unified Visual Multi-modal Object Trackingcited by 11
  11. 2025CrossLMM: Decoupling Long Video Sequences from LMMs via Dual Cross-Attention Mechanismscited by 11
  12. 2025CADTrack: Learning Contextual Aggregation with Deformable Alignment for Robust RGBT Trackingcited by 11
38 more
  1. 2024General Compression Framework for Efficient Transformer Object Trackingcited by 10
  2. 2025CSTrack: Enhancing RGB-X Tracking via Compact Spatiotemporal Featurescited by 10
  3. 2025ATCTrack: Aligning Target-Context Cues with Dynamic Target States for Robust Vision-Language Trackingcited by 10
  4. 2024Improving Visual Object Tracking through Visual Promptingcited by 9
  5. 2025Temporal Unlearnable Examples: Preventing Personal Video Data from Unauthorized Exploitation by Object Trackingcited by 9
  6. 2025Controllable-LPMoE: Adapting to Challenging Object Segmentation via Dynamic Local Priors from Mixture-of-Expertscited by 9
  7. 2024Enhancing Vision-Language Tracking by Effectively Converting Textual Cues into Visual Cuescited by 8
  8. 2024AFter: Attention-based Fusion Router for RGBT Trackingcited by 7
  9. 2024X-Prompt: Multi-modal Visual Prompt for Video Object Segmentationcited by 7
  10. 2025DeTrack: In-model Latent Denoising Learning for Visual Object Trackingcited by 5
  11. 2025SAMITE: Position Prompted SAM2 with Calibrated Memory for Visual Object Trackingcited by 5
  12. 2026Video Understanding: From Geometry and Semantics to Unified Modelscited by 5
  13. 2025Correlative and Discriminative Label Grouping for Multi-Label Visual Prompt Tuningcited by 4
  14. 2025Collaborating Vision, Depth, and Thermal Signals for Multi-Modal Tracking: Dataset and Algorithmcited by 3
  15. 2025VPTracker: Global Vision-Language Tracking via Visual Promptcited by 3
  16. 2026SEATrack: Simple, Efficient, and Adaptive Multimodal Trackercited by 3
  17. 2026RELO: Reinforcement Learning to Localize for Visual Object Trackingcited by 3
  18. 2024Cross Fusion RGB-T Tracking with Bi-directional Adaptercited by 2
  19. 2025Breaking Shallow Limits: Task-Driven Pixel Fusion for Gap-free RGBT Trackingcited by 2
  20. 2025Diff-MM: Exploring Pre-trained Text-to-Image Generation Model for Unified Multi-modal Object Trackingcited by 2
  21. 2025Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Trackingcited by 2
  22. 2025Omni Survey for Multimodality Analysis in Visual Object Trackingcited by 2
  23. 2026FARTrack: Fast Autoregressive Visual Tracking with High Performancecited by 2
  24. 2026UETrack: A Unified and Efficient Framework for Single Object Trackingcited by 2
  25. 2026RAGTrack: Language-aware RGBT Tracking with Retrieval-Augmented Generationcited by 2
  26. 2024Middle Fusion and Multi-Stage, Multi-Form Prompts for Robust RGB-T Trackingcited by 1
  27. 2025Boosting Adversarial Transferability with Spatial Adversarial Alignmentcited by 1
  28. 2025UASTrack: A Unified Adaptive Selection Framework with Modality-Customization in Single Object Trackingcited by 1
  29. 2025Progressive Scaling Visual Object Trackingcited by 1
  30. 2025Learning Frequency and Memory-Aware Prompts for Multi-Modal Object Trackingcited by 1
  31. 2025DMTrack: Spatio-Temporal Multimodal Tracking via Dual-Adaptercited by 1
  32. 2025SwiTrack: Tri-State Switch for Cross-Modal Object Trackingcited by 1
  33. 2025Tracking and Segmenting Anything in Any Modalitycited by 1
  34. 2026Detector-Augmented SAMURAI for Long-Duration Drone Trackingcited by 1
  35. 2026Uni-MDTrack: Learning Decoupled Memory and Dynamic States for Parameter-Efficient Visual Tracking in All Modalitycited by 1
  36. 2026Unified Multimodal Visual Tracking with Dual Mixture-of-Expertscited by 1
  37. 2026Spatio-Temporal Conditional Denoising Transformer for Modality-Missing RGBT Trackingcited by 1
  38. 2025Hyperspectral Adapter for Object Tracking based on Hyperspectral Videocited by 0

50 most cited of 65 All 65 (JSON)