# RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss

arXiv: https://arxiv.org/abs/2011.07189 · HTML: https://mlsanity.com/paper/2011.07189/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2020.

## Built on by (most cited first, 30 of 32)

- [LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking](https://mlsanity.com/paper/2104.13202/) (2021)
- [Bi-directional Adapter for Multi-modal Tracking](https://mlsanity.com/paper/2312.10611/) (2023)
- [Does Thermal Really Always Matter for RGB-T Salient Object Detection?](https://mlsanity.com/paper/2210.04266/) (2022)
- [CAFuser: Condition-Aware Multimodal Fusion for Robust Semantic Perception of Driving Scenes](https://mlsanity.com/paper/2410.10791/) (2024)
- [Temporal Adaptive RGBT Tracking with Modality Prompt](https://mlsanity.com/paper/2401.01244/) (2024)
- [Multi-spectral Vehicle Re-identification with Cross-directional Consistency Network and a High-quality Benchmark](https://mlsanity.com/paper/2208.00632/) (2022)
- [Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks](https://mlsanity.com/paper/2312.16244/) (2023)
- [Cross-modulated Attention Transformer for RGBT Tracking](https://mlsanity.com/paper/2408.02222/) (2024)
- [Transformer RGBT Tracking with Spatio-Temporal Multimodal Tokens](https://mlsanity.com/paper/2401.01674/) (2024)
- [Breaking Modality Gap in RGBT Tracking: Coupled Knowledge Distillation](https://mlsanity.com/paper/2410.11586/) (2024)
- [RGB-T Tracking via Multi-Modal Mutual Prompt Learning](https://mlsanity.com/paper/2308.16386/) (2023)
- [ICPL-ReID: Identity-Conditional Prompt Learning for Multi-Spectral Object Re-Identification](https://mlsanity.com/paper/2505.17821/) (2025)
- [RGB-T Tracking Based on Mixed Attention](https://mlsanity.com/paper/2304.04264/) (2023)
- [AFter: Attention-based Fusion Router for RGBT Tracking](https://mlsanity.com/paper/2405.02717/) (2024)
- [A Survey for Deep RGBT Tracking](https://mlsanity.com/paper/2201.09296/) (2022)
- [Unified Single-Stage Transformer Network for Efficient RGB-T Tracking](https://mlsanity.com/paper/2308.13764/) (2023)
- [RGBT Tracking via Progressive Fusion Transformer with Dynamically Guided Learning](https://mlsanity.com/paper/2303.14778/) (2023)
- [From Two-Stream to One-Stream: Efficient RGB-T Tracking via Mutual Prompt Learning and Knowledge Distillation](https://mlsanity.com/paper/2403.16834/) (2024)
- [Breaking Shallow Limits: Task-Driven Pixel Fusion for Gap-free RGBT Tracking](https://mlsanity.com/paper/2503.11247/) (2025)
- [Omni Survey for Multimodality Analysis in Visual Object Tracking](https://mlsanity.com/paper/2508.13000/) (2025)
- [MTNet: Learning modality-aware representation with transformer for RGBT tracking](https://mlsanity.com/paper/2508.17280/) (2025)
- [RAGTrack: Language-aware RGBT Tracking with Retrieval-Augmented Generation](https://mlsanity.com/paper/2603.03617/) (2026)
- [Middle Fusion and Multi-Stage, Multi-Form Prompts for Robust RGB-T Tracking](https://mlsanity.com/paper/2403.18193/) (2024)
- [BTMTrack: Robust RGB-T Tracking via Dual-template Bridging and Temporal-Modal Candidate Elimination](https://mlsanity.com/paper/2501.03616/) (2025)
- [LightFC-X: Lightweight Convolutional Tracker for RGB-X Tracking](https://mlsanity.com/paper/2502.18143/) (2025)
- [UASTrack: A Unified Adaptive Selection Framework with Modality-Customization in Single Object Tracking](https://mlsanity.com/paper/2502.18220/) (2025)
- [Spatio-Temporal Conditional Denoising Transformer for Modality-Missing RGBT Tracking](https://mlsanity.com/paper/2607.24701/) (2026)
- [Self-Supervised RGB-T Tracking with Cross-Input Consistency](https://mlsanity.com/paper/2301.11274/) (2023)
- [EANet: Enhanced Attribute-based RGBT Tracker Network](https://mlsanity.com/paper/2307.01893/) (2023)
- [Revisiting RGBT Tracking Benchmarks from the Perspective of Modality Validity: A New Benchmark, Problem, and Solution](https://mlsanity.com/paper/2405.00168/) (2024)

## Abstract (arXiv)

RGBT tracking has attracted increasing attention since RGB and thermal infrared data have strong complementary advantages, which could make trackers all-day and all-weather work. However, how to effectively represent RGBT data for visual tracking remains unstudied well. Existing works usually focus on extracting modality-shared or modality-specific information, but the potentials of these two cues are not well explored and exploited in RGBT tracking. In this paper, we propose a novel multi-adapter network to jointly perform modality-shared, modality-specific and instance-aware target representation learning for RGBT tracking. To this end, we design three kinds of adapters within an end-to-end deep learning framework. In specific, we use the modified VGG-M as the generality adapter to extract the modality-shared target representations.To extract the modality-specific features while reducing the computational complexity, we design a modality adapter, which adds a small block to the generality adapter in each layer and each modality in a parallel manner. Such a design could learn multilevel modality-specific representations with a modest number of parameters as the vast majority of parameters are shared with the generality adapter. We also design instance adapter to capture the appearance properties and temporal variations of a certain target. Moreover, to enhance the shared and specific features, we employ the loss of multiple kernel maximum mean discrepancy to measure the distribution divergence of different modal features and integrate it into each layer for more robust representation learning. Extensive experiments on two RGBT tracking benchmark datasets demonstrate the outstanding performance of the proposed tracker against the state-of-the-art methods.