mlsanityData to 14 Oct 2026

Prototype-based Cross-Modal Object Tracking

arXiv 2312.14471 ↗First posted 22 Dec 2023cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Cross-modal object tracking is an important research topic in the field of information fusion, and it aims to address imaging limitations in challenging scenarios by integrating switchable visible and near-infrared modalities. However, existing tracking methods face some difficulties in adapting to significant target appearance variations in the presence of modality switch. For instance, model update based tracking methods struggle to maintain stable tracking results during modality switching, leading to error accumulation and model drift. Template based tracking methods solely rely on the template information from first frame and/or last frame, which lacks sufficient representation ability and poses challenges in handling significant target appearance changes. To address this problem, we propose a prototype-based cross-modal object tracker called ProtoTrack, which introduces a novel prototype learning scheme to adapt to significant target appearance variations, for cross-modal object tracking. In particular, we design a multi-modal prototype to represent target information by multi-kind samples, including a fixed sample from the first frame and two representative samples from different modalities. Moreover, we develop a prototype generation algorithm based on two new modules to ensure the prototype representative in different challenges......

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 50 of its 62 references that are in the map. First posted on arXiv 22 Dec 2023.

Builds on

50 of 62 references.

References · 50

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  6. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 568
  7. 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 475
  8. 2019PANet: Few-Shot Image Semantic Segmentation with Prototype Alignmentcited by 449
  9. 2018SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networkscited by 446
  10. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 429
  11. 2021Transformer Trackingcited by 412
  12. 2021Learning Spatio-Temporal Transformer for Visual Trackingcited by 350
38 more
  1. 2019Learning Discriminative Model Prediction for Trackingcited by 344
  2. 2018TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wildcited by 343
  3. 2022Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Frameworkcited by 330
  4. 2018ATOM: Accurate Tracking by Overlap Maximizationcited by 285
  5. 2022MixFormer: End-to-End Tracking with Iterative Mixed Attentioncited by 281
  6. 2018Fast Online Object Tracking and Segmentation: A Unifying Approachcited by 272
  7. 2020Probabilistic Regression for Visual Trackingcited by 231
  8. 2021Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Trackingcited by 220
  9. 2019CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learningcited by 217
  10. 2020Ocean: Object-aware Anchor-free Trackingcited by 212
  11. 2015Learning Multi-Domain Convolutional Neural Networks for Visual Trackingcited by 209
  12. 2021Adaptive Prototype Learning and Allocation for Few-Shot Segmentationcited by 207
  13. 2020Siamese Box Adaptive Network for Visual Trackingcited by 189
  14. 2019Meta R-CNN : Towards General Solver for Instance-level Few-shot Learningcited by 182
  15. 2018Robust Classification with Convolutional Prototype Learningcited by 123
  16. 2018RGB-T Object Tracking:Benchmark and Baselinecited by 121
  17. 2021Diverse Part Discovery: Occluded Person Re-identification with Part-Aware Transformercited by 120
  18. 2022Transforming Model Prediction for Trackingcited by 117
  19. 2020CRNet: Cross-Reference Networks for Few-Shot Segmentationcited by 101
  20. 2021LasHeR: A Large-scale High-diversity Benchmark for RGBT Trackingcited by 96
  21. 2022Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baselinecited by 91
  22. 2020Revisiting the Sibling Head in Object Detectorcited by 82
  23. 2020High-Performance Long-Term Tracking with Meta-Updatercited by 80
  24. 2019Learning the Model Update for Siamese Trackerscited by 76
  25. 2016Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Trackingcited by 66
  26. 2017Learning Background-Aware Correlation Filters for Visual Trackingcited by 65
  27. 2018Learning Dynamic Memory Networks for Object Trackingcited by 59
  28. 2019GlobalTrack: A Simple and Strong Baseline for Long-term Trackingcited by 57
  29. 2023PRIOR: Prototype Representation Joint Learning from Medical Images and Reportscited by 55
  30. 2018Real-Time MDNetcited by 39
  31. 2022Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentationcited by 37
  32. 2022Dynamic Prototype Mask for Occluded Person Re-Identificationcited by 30
  33. 2018Revisiting RCNN: On Awakening the Classification Power of Faster RCNNcited by 28
  34. 2022Exploring Fusion Strategies for Accurate RGBT Visual Object Trackingcited by 26
  35. 2023TranSG: Transformer-Based Skeleton Graph Prototype Contrastive Learning with Structure-Trajectory Prompted Reconstruction for Person Re-Identificationcited by 14
  36. 2018Context-aware Deep Feature Compression for High-speed Visual Trackingcited by 8
  37. 2021Cross-Modal Object Tracking: Modality-Aware Representations and A Unified Benchmarkcited by 7
  38. 2016Deep Learning Prototype Domains for Person Re-Identificationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025Cognitive Disentanglement for Referring Multi-Object Trackingcited by 6
  2. 2025COST: Contrastive One-Stage Transformer for Vision-Language Small Object Trackingcited by 2
  3. 2025Omni Survey for Multimodality Analysis in Visual Object Trackingcited by 2
  4. 2025SwiTrack: Tri-State Switch for Cross-Modal Object Trackingcited by 1
  5. 2026Unified Multimodal Visual Tracking with Dual Mixture-of-Expertscited by 1
  6. 2026Cross-Modal UAV Object Tracking: State-Aware Representation Learning and A Unified Benchmarkcited by 0
  7. 2026Template-Search Domain Adaptation via Multi-Stage Feature Alignment for Cross-Modal Object Trackingcited by 0