# MTNet: Learning modality-aware representation with transformer for RGBT tracking

arXiv: https://arxiv.org/abs/2508.17280 · HTML: https://mlsanity.com/paper/2508.17280/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 24 Aug 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Learning Spatio-Temporal Transformer for Visual Tracking](https://mlsanity.com/paper/2103.17154/) (2021)
- [RGB-T Object Tracking:Benchmark and Baseline](https://mlsanity.com/paper/1805.08982/) (2018)
- [LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking](https://mlsanity.com/paper/2104.13202/) (2021)
- [Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline](https://mlsanity.com/paper/2204.04120/) (2022)
- [Multi-Modal Fusion for End-to-End RGB-T Tracking](https://mlsanity.com/paper/1908.11714/) (2019)
- [Real-Time MDNet](https://mlsanity.com/paper/1808.08834/) (2018)
- [Duality-Gated Mutual Condition Network for RGBT Tracking](https://mlsanity.com/paper/2011.07188/) (2020)
- [RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss](https://mlsanity.com/paper/2011.07189/) (2020)
- [MFGNet: Dynamic Modality-Aware Filter Generation for RGB-T Tracking](https://mlsanity.com/paper/2107.10433/) (2021)

## Built on by (most cited first, 2 of 2)

- [SwiTrack: Tri-State Switch for Cross-Modal Object Tracking](https://mlsanity.com/paper/2511.16227/) (2025)
- [Template-Search Domain Adaptation via Multi-Stage Feature Alignment for Cross-Modal Object Tracking](https://mlsanity.com/paper/2609.38637/) (2026)

## Abstract (arXiv)

The ability to learn robust multi-modality representation has played a critical role in the development of RGBT tracking. However, the regular fusion paradigm and the invariable tracking template remain restrictive to the feature interaction. In this paper, we propose a modality-aware tracker based on transformer, termed MTNet. Specifically, a modality-aware network is presented to explore modality-specific cues, which contains both channel aggregation and distribution module(CADM) and spatial similarity perception module (SSPM). A transformer fusion network is then applied to capture global dependencies to reinforce instance representations. To estimate the precise location and tackle the challenges, such as scale variation and deformation, we design a trident prediction head and a dynamic update strategy which jointly maintain a reliable template for facilitating inter-frame communication. Extensive experiments validate that the proposed method achieves satisfactory results compared with the state-of-the-art competitors on three RGBT benchmarks while reaching real-time speed.