# Dynamic Fusion Network for RGBT Tracking

arXiv: https://arxiv.org/abs/2109.07662 · HTML: https://mlsanity.com/paper/2109.07662/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 65 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Builds on (most cited first, 19 of 19)

- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [CondConv: Conditionally Parameterized Convolutions for Efficient Inference](https://mlsanity.com/paper/1904.04971/) (2019)
- [Learning Multi-Domain Convolutional Neural Networks for Visual Tracking](https://mlsanity.com/paper/1510.07945/) (2015)
- [Acquisition of Localization Confidence for Accurate Object Detection](https://mlsanity.com/paper/1807.11590/) (2018)
- [Return of the Devil in the Details: Delving Deep into Convolutional Nets](https://mlsanity.com/paper/1405.3531/) (2014)
- [RGB-T Object Tracking:Benchmark and Baseline](https://mlsanity.com/paper/1805.08982/) (2018)
- [Deep Learning for Visual Tracking: A Comprehensive Survey](https://mlsanity.com/paper/1912.00535/) (2019)
- [Challenge-Aware RGBT Tracking](https://mlsanity.com/paper/2007.13143/) (2020)
- [WeightNet: Revisiting the Design Space of Weight Networks](https://mlsanity.com/paper/2007.11823/) (2020)
- [Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking](https://mlsanity.com/paper/2007.02041/) (2020)
- [Real-Time MDNet](https://mlsanity.com/paper/1808.08834/) (2018)
- [Multi-Adapter RGBT Tracking](https://mlsanity.com/paper/1907.07485/) (2019)
- [Studying Very Low Resolution Recognition Using Deep Networks](https://mlsanity.com/paper/1601.04153/) (2016)
- [Synthetic data generation for end-to-end thermal infrared tracking](https://mlsanity.com/paper/1806.01013/) (2018)
- [Siamese Infrared and Visible Light Fusion Network for RGB-T Tracking](https://mlsanity.com/paper/2103.07302/) (2021)
- [Hierarchical Spatial-aware Siamese Network for Thermal Infrared Object Tracking](https://mlsanity.com/paper/1711.09539/) (2017)
- [Free-Form Image Inpainting via Contrastive Attention Network](https://mlsanity.com/paper/2010.15643/) (2020)

## Built on by (most cited first, 9 of 9)

- [RGBT Tracking via All-layer Multimodal Interactions with Progressive Fusion Mamba](https://mlsanity.com/paper/2408.08827/) (2024)
- [Breaking Modality Gap in RGBT Tracking: Coupled Knowledge Distillation](https://mlsanity.com/paper/2410.11586/) (2024)
- [AFter: Attention-based Fusion Router for RGBT Tracking](https://mlsanity.com/paper/2405.02717/) (2024)
- [RGBT Tracking via Progressive Fusion Transformer with Dynamically Guided Learning](https://mlsanity.com/paper/2303.14778/) (2023)
- [HDRT: A Large-Scale Dataset for Infrared-Guided HDR Imaging](https://mlsanity.com/paper/2406.05475/) (2024)
- [Optimizing Multispectral Object Detection: A Bag of Tricks and Comprehensive Benchmarks](https://mlsanity.com/paper/2411.18288/) (2024)
- [X Modality Assisting RGBT Object Tracking](https://mlsanity.com/paper/2312.17273/) (2023)
- [Visual Object Tracking across Diverse Data Modalities: A Review](https://mlsanity.com/paper/2412.09991/) (2024)
- [CapHDR2IR: Caption-Driven Transfer from Visible Light to Infrared Domain](https://mlsanity.com/paper/2411.16327/) (2024)

## Abstract (arXiv)

For both visible and infrared images have their own advantages and disadvantages, RGBT tracking has attracted more and more attention. The key points of RGBT tracking lie in feature extraction and feature fusion of visible and infrared images. Current RGBT tracking methods mostly pay attention to both individual features (features extracted from images of a single camera) and common features (features extracted and fused from an RGB camera and a thermal camera), while pay less attention to the different and dynamic contributions of individual features and common features for different sequences of registered image pairs. This paper proposes a novel RGBT tracking method, called Dynamic Fusion Network (DFNet), which adopts a two-stream structure, in which two non-shared convolution kernels are employed in each layer to extract individual features. Besides, DFNet has shared convolution kernels for each layer to extract common features. Non-shared convolution kernels and shared convolution kernels are adaptively weighted and summed according to different image pairs, so that DFNet can deal with different contributions for different sequences. DFNet has a fast speed, which is 28.658 FPS. The experimental results show that when DFNet only increases the Mult-Adds of 0.02% than the non-shared-convolution-kernel-based fusion method, Precision Rate (PR) and Success Rate (SR) reach 88.1% and 71.9% respectively.