mlsanityData to 7 Oct 2026

Prompting for Multi-Modal Tracking

arXiv 2207.14571 ↗First posted 29 Jul 2022cs.CVcited by 58

Not on the map. Why?

Abstract (arXiv)

Multi-modal tracking gains attention due to its ability to be more accurate and robust in complex scenarios compared to traditional RGB-based tracking. Its key lies in how to fuse multi-modal data and reduce the gap between modalities. However, multi-modal tracking still severely suffers from data deficiency, thus resulting in the insufficient learning of fusion modules. Instead of building such a fusion module, in this paper, we provide a new perspective on multi-modal tracking by attaching importance to the multi-modal visual prompts. We design a novel multi-modal prompt tracker (ProTrack), which can transfer the multi-modal inputs to a single modality by the prompt paradigm. By best employing the tracking ability of pre-trained RGB trackers learning at scale, our ProTrack can achieve high-performance multi-modal tracking by only altering the inputs, even without any extra training on multi-modal data. Extensive experiments on 5 benchmark datasets demonstrate the effectiveness of the proposed ProTrack.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Builds on 29 of its 39 references that are in the map. First posted on arXiv 29 Jul 2022.

Builds on

29 of 39 references.

References · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  5. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2022Visual Prompt Tuningcited by 1,674
  8. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  9. 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 471
  10. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 424
  11. 2021Transformer Trackingcited by 412
  12. 2021Learning Spatio-Temporal Transformer for Visual Trackingcited by 348
17 more
  1. 2019Learning Discriminative Model Prediction for Trackingcited by 344
  2. 2018TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wildcited by 342
  3. 2018ATOM: Accurate Tracking by Overlap Maximizationcited by 285
  4. 2020Probabilistic Regression for Visual Trackingcited by 231
  5. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Modelscited by 182
  6. 2021Learning Target Candidate Association to Keep Track of What Not to Trackcited by 131
  7. 2021VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flowscited by 128
  8. 2018RGB-T Object Tracking:Benchmark and Baselinecited by 121
  9. 2021LasHeR: A Large-scale High-diversity Benchmark for RGBT Trackingcited by 94
  10. 2020Alpha-Refine: Boosting Tracking Performance by Precise Bounding Box Estimationcited by 81
  11. 2021DepthTrack : Unveiling the Power of RGBD Trackingcited by 66
  12. 2019Multi-Modal Fusion for End-to-End RGB-T Trackingcited by 63
  13. 2018Object Tracking by Reconstruction with View-Specific Discriminative Correlation Filterscited by 32
  14. 2019CDTB: A Color and Depth Visual Object Tracking Dataset and Benchmarkcited by 31
  15. 2019DAL -- A Deep Depth-aware Long-term Trackercited by 25
  16. 2020Asynchronous Tracking-by-Detection on Adaptive Time Surfaces for Event-based Object Trackingcited by 25
  17. 2020TSDM: Tracking by SiamRPN++ with a Depth-refiner and a Mask-generatorcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 9; 2024: 18; 2025: 18; 2026: 12.

58 papers.

Sort:

By area:

  1. 2023Visual Prompt Multi-Modal Trackingcited by 141
  2. 2023Multimodal Prompting with Missing Modalities for Visual Recognitioncited by 121
  3. 2023Single-Model and Any-Modality for Video Object Trackingcited by 64
  4. 2024OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuningcited by 64
  5. 2022Dual Modality Prompt Tuning for Vision-Language Pre-Trained Modelcited by 51
  6. 2024SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Trackingcited by 43
  7. 2023Generative-based Fusion Mechanism for Multi-Modal Trackingcited by 33
  8. 2025Towards Universal Modal Tracking with Online Dense Temporal Token Learningcited by 30
  9. 2023Efficient Multimodal Semantic Segmentation via Dual-Prompt Learningcited by 25
  10. 2023Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Trackingcited by 20
  11. 2024XTrack: Multimodal Training Boosts RGB-X Video Object Trackerscited by 20
  12. 2023Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarkscited by 17
38 more
  1. 2024CRSOT: Cross-Resolution Object Tracking using Unaligned Frame and Event Camerascited by 17
  2. 2024Autogenic Language Embedding for Coherent Point Trackingcited by 17
  3. 2024RGBT Tracking via All-layer Multimodal Interactions with Progressive Fusion Mambacited by 17
  4. 2025What You Have is What You Track: Adaptive and Robust Multimodal Trackingcited by 15
  5. 2024MambaVT: Spatio-Temporal Contextual Modeling for robust RGB-T Trackingcited by 14
  6. 2024Breaking Modality Gap in RGBT Tracking: Coupled Knowledge Distillationcited by 11
  7. 2025Adaptive Perception for Unified Visual Multi-modal Object Trackingcited by 11
  8. 2024POV: Prompt-Oriented View-Agnostic Learning for Egocentric Hand-Object Interaction in the Multi-View Worldcited by 10
  9. 2025CSTrack: Enhancing RGB-X Tracking via Compact Spatiotemporal Featurescited by 10
  10. 2023Visual Prompt Flexible-Modal Face Anti-Spoofingcited by 9
  11. 2024TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling and Mutually-Guided Fusion for RGB-E Object Trackingcited by 9
  12. 2024Reliable Object Tracking by Multimodal Hybrid Feature Extraction and Transformer-Based Fusioncited by 9
  13. 2024AFter: Attention-based Fusion Router for RGBT Trackingcited by 7
  14. 2024X-Prompt: Multi-modal Visual Prompt for Video Object Segmentationcited by 7
  15. 2024SSFam: Scribble Supervised Salient Object Detection Familycited by 6
  16. 2026Video Understanding: From Geometry and Semantics to Unified Modelscited by 5
  17. 2024Transformer-based RGB-T Tracking with Channel and Spatial Feature Fusioncited by 4
  18. 2025Efficient Motion Prompt Learning for Robust Visual Trackingcited by 3
  19. 2024Proactive Schemes: A Survey of Adversarial Attacks for Social Goodcited by 2
  20. 2025Breaking Shallow Limits: Task-Driven Pixel Fusion for Gap-free RGBT Trackingcited by 2
  21. 2025Diff-MM: Exploring Pre-trained Text-to-Image Generation Model for Unified Multi-modal Object Trackingcited by 2
  22. 2025Omni Survey for Multimodality Analysis in Visual Object Trackingcited by 2
  23. 2026UETrack: A Unified and Efficient Framework for Single Object Trackingcited by 2
  24. 2026SEATrack: Simple, Efficient, and Adaptive Multimodal Trackercited by 2
  25. 2023Towards Subject-Oriented Video Captioning via User-Specified Targetscited by 1
  26. 2024Middle Fusion and Multi-Stage, Multi-Form Prompts for Robust RGB-T Trackingcited by 1
  27. 2025LightFC-X: Lightweight Convolutional Tracker for RGB-X Trackingcited by 1
  28. 2025UASTrack: A Unified Adaptive Selection Framework with Modality-Customization in Single Object Trackingcited by 1
  29. 2025Towards General Multimodal Visual Trackingcited by 1
  30. 2025Progressive Scaling Visual Object Trackingcited by 1
  31. 2025Learning Frequency and Memory-Aware Prompts for Multi-Modal Object Trackingcited by 1
  32. 2025DMTrack: Spatio-Temporal Multimodal Tracking via Dual-Adaptercited by 1
  33. 2025UniSOT: A Unified Framework for Multi-Modality Single Object Trackingcited by 1
  34. 2025SwiTrack: Tri-State Switch for Cross-Modal Object Trackingcited by 1
  35. 2026Uni-MDTrack: Learning Decoupled Memory and Dynamic States for Parameter-Efficient Visual Tracking in All Modalitycited by 1
  36. 2026Spatio-Temporal Conditional Denoising Transformer for Modality-Missing RGBT Trackingcited by 1
  37. 2024Revisiting RGBT Tracking Benchmarks from the Perspective of Modality Validity: A New Benchmark, Problem, and Solutioncited by 0
  38. 2025Hyperspectral Adapter for Object Tracking based on Hyperspectral Videocited by 0

50 most cited of 58 All 58 (JSON)