# Exploring Fusion Strategies for Accurate RGBT Visual Object Tracking

arXiv: https://arxiv.org/abs/2201.08673 · HTML: https://mlsanity.com/paper/2201.08673/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 84 references that are in the map.
- First posted on arXiv 21 Jan 2022.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016)
- [YouTube-VOS: Sequence-to-Sequence Video Object Segmentation](https://mlsanity.com/paper/1809.00461/) (2018)
- [Deeper and Wider Siamese Networks for Real-Time Visual Tracking](https://mlsanity.com/paper/1901.01660/) (2019)
- [RGB-T Object Tracking:Benchmark and Baseline](https://mlsanity.com/paper/1805.08982/) (2018)
- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking](https://mlsanity.com/paper/2104.13202/) (2021)
- [Learning the Model Update for Siamese Trackers](https://mlsanity.com/paper/1908.00855/) (2019)
- [Learning Spatially Regularized Correlation Filters for Visual Tracking](https://mlsanity.com/paper/1608.05571/) (2016)
- [Siamese Cascaded Region Proposal Networks for Real-Time Visual Tracking](https://mlsanity.com/paper/1812.06148/) (2018)
- [Multi-Modal Fusion for End-to-End RGB-T Tracking](https://mlsanity.com/paper/1908.11714/) (2019)
- [MDLatLRR: A novel decomposition method for infrared and visible image fusion](https://mlsanity.com/paper/1811.02291/) (2018)
- [Challenge-Aware RGBT Tracking](https://mlsanity.com/paper/2007.13143/) (2020)
- [Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking](https://mlsanity.com/paper/2007.02041/) (2020)
- [Dense Feature Aggregation and Pruning for RGBT Tracking](https://mlsanity.com/paper/1907.10451/) (2019)
- [Multi-Adapter RGBT Tracking](https://mlsanity.com/paper/1907.07485/) (2019)
- [Synthetic data generation for end-to-end thermal infrared tracking](https://mlsanity.com/paper/1806.01013/) (2018)
- [Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Tracking](https://mlsanity.com/paper/1807.11348/) (2018)
- [Learning Deep Multi-Level Similarity for Thermal Infrared Object Tracking](https://mlsanity.com/paper/1906.03568/) (2019)
- [Joint Group Feature Selection and Discriminative Filter Learning for Robust Visual Object Tracking](https://mlsanity.com/paper/1907.13242/) (2019)

## Built on by (most cited first, 26 of 26)

- [LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images](https://mlsanity.com/paper/2304.05172/) (2023)
- [Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption](https://mlsanity.com/paper/2501.10761/) (2025)
- [Generative-based Fusion Mechanism for Multi-Modal Tracking](https://mlsanity.com/paper/2309.01728/) (2023)
- [One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion](https://mlsanity.com/paper/2502.19854/) (2025)
- [Visible-Thermal Multiple Object Tracking: Large-scale Video Dataset and Progressive Fusion Approach](https://mlsanity.com/paper/2408.00969/) (2024)
- [FusionBooster: A Unified Image Fusion Boosting Paradigm](https://mlsanity.com/paper/2305.05970/) (2023)
- [Breaking Modality Gap in RGBT Tracking: Coupled Knowledge Distillation](https://mlsanity.com/paper/2410.11586/) (2024)
- [TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling and Mutually-Guided Fusion for RGB-E Object Tracking](https://mlsanity.com/paper/2405.05004/) (2024)
- [Prototype-based Cross-Modal Object Tracking](https://mlsanity.com/paper/2312.14471/) (2023)
- [AFter: Attention-based Fusion Router for RGBT Tracking](https://mlsanity.com/paper/2405.02717/) (2024)
- [RGBT Tracking via Progressive Fusion Transformer with Dynamically Guided Learning](https://mlsanity.com/paper/2303.14778/) (2023)
- [Transformer-based RGB-T Tracking with Channel and Spatial Feature Fusion](https://mlsanity.com/paper/2405.03177/) (2024)
- [Optimizing Multispectral Object Detection: A Bag of Tricks and Comprehensive Benchmarks](https://mlsanity.com/paper/2411.18288/) (2024)
- [Explicit Context Reasoning with Supervision for Visual Tracking](https://mlsanity.com/paper/2507.16191/) (2025)
- [Dynamic Subframe Splitting and Spatio-Temporal Motion Entangled Sparse Attention for RGB-E Tracking](https://mlsanity.com/paper/2409.17560/) (2024)
- [Breaking Shallow Limits: Task-Driven Pixel Fusion for Gap-free RGBT Tracking](https://mlsanity.com/paper/2503.11247/) (2025)
- [GrFormer: A Novel Transformer on Grassmann Manifold for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2506.14384/) (2025)
- [Omni Survey for Multimodality Analysis in Visual Object Tracking](https://mlsanity.com/paper/2508.13000/) (2025)
- [Motion State: A New Benchmark Multiple Object Tracking](https://mlsanity.com/paper/2312.17641/) (2023)
- [UASTrack: A Unified Adaptive Selection Framework with Modality-Customization in Single Object Tracking](https://mlsanity.com/paper/2502.18220/) (2025)
- [RGB-D Video Object Segmentation via Enhanced Multi-store Feature Memory](https://mlsanity.com/paper/2504.16471/) (2025)
- [Revisiting RGBT Tracking Benchmarks from the Perspective of Modality Validity: A New Benchmark, Problem, and Solution](https://mlsanity.com/paper/2405.00168/) (2024)
- [Addressing single object tracking in satellite imagery through prompt-engineered solutions](https://mlsanity.com/paper/2407.05518/) (2024)
- [Predicting the Best of N Visual Trackers](https://mlsanity.com/paper/2407.15707/) (2024)
- [Serial Over Parallel: Learning Continual Unification for Multi-Modal Visual Object Tracking and Benchmarking](https://mlsanity.com/paper/2508.10655/) (2025)
- [Learning Progressive Adaptation for Multi-Modal Tracking](https://mlsanity.com/paper/2603.21100/) (2026)

## Abstract (arXiv)

We address the problem of multi-modal object tracking in video and explore various options of fusing the complementary information conveyed by the visible (RGB) and thermal infrared (TIR) modalities including pixel-level, feature-level and decision-level fusion. Specifically, different from the existing methods, paradigm of image fusion task is heeded for fusion at pixel level. Feature-level fusion is fulfilled by attention mechanism with channels excited optionally. Besides, at decision level, a novel fusion strategy is put forward since an effortless averaging configuration has shown the superiority. The effectiveness of the proposed decision-level fusion strategy owes to a number of innovative contributions, including a dynamic weighting of the RGB and TIR contributions and a linear template update operation. A variant of which produced the winning tracker at the Visual Object Tracking Challenge 2020 (VOT-RGBT2020). The concurrent exploration of innovative pixel- and feature-level fusion strategies highlights the advantages of the proposed decision-level fusion method. Extensive experimental results on three challenging datasets, textiti.e., GTOT, VOT-RGBT2019, and VOT-RGBT2020, demonstrate the effectiveness and robustness of the proposed method, compared to the state-of-the-art approaches. Code will be shared at textcolorbluehttps://github.com/Zhangyong-Tang/DFAT.