# Robust RGB-T Tracking via Learnable Visual Fourier Prompt Fine-tuning and Modality Fusion Prompt Generation

arXiv: https://arxiv.org/abs/2509.19733 · HTML: https://mlsanity.com/paper/2509.19733/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 63 of its 95 references that are in the map.
- First posted on arXiv 24 Sep 2025.

## Builds on (most cited first, 30 of 63)

- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks](https://mlsanity.com/paper/1905.13545/) (2019)
- [How Do Vision Transformers Work?](https://mlsanity.com/paper/2202.06709/) (2022)
- [MACE: Mass Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2403.06135/) (2024)
- [Global Filter Networks for Image Classification](https://mlsanity.com/paper/2107.00645/) (2021)
- [TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition](https://mlsanity.com/paper/2307.12493/) (2023)
- [Visual Prompt Multi-Modal Tracking](https://mlsanity.com/paper/2303.10826/) (2023)
- [Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice](https://mlsanity.com/paper/2203.05962/) (2022)
- [ODTrack: Online Dense Temporal Token Learning for Visual Tracking](https://mlsanity.com/paper/2401.01686/) (2024)
- [RGB-T Object Tracking:Benchmark and Baseline](https://mlsanity.com/paper/1805.08982/) (2018)
- [Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances](https://mlsanity.com/paper/2410.18775/) (2024)
- [Inception Transformer](https://mlsanity.com/paper/2205.12956/) (2022)
- [LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking](https://mlsanity.com/paper/2104.13202/) (2021)
- [EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers](https://mlsanity.com/paper/2412.20413/) (2024)
- [Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping](https://mlsanity.com/paper/2305.11003/) (2023)
- [Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline](https://mlsanity.com/paper/2204.04120/) (2022)
- [Diffusion Models in Low-Level Vision: A Survey](https://mlsanity.com/paper/2406.11138/) (2024)
- [Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model](https://mlsanity.com/paper/2311.11638/) (2023)
- [Single-Model and Any-Modality for Video Object Tracking](https://mlsanity.com/paper/2311.15851/) (2023)
- [Multi-Modal Fusion for End-to-End RGB-T Tracking](https://mlsanity.com/paper/1908.11714/) (2019)
- [Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Concepts](https://mlsanity.com/paper/2504.12782/) (2025)
- [Person Re-identification Method Based on Color Attack and Joint Defence](https://mlsanity.com/paper/2111.09571/) (2021)
- [Strategic Preys Make Acute Predators: Enhancing Camouflaged Object Detectors by Generating Camouflaged Objects](https://mlsanity.com/paper/2308.03166/) (2023)
- [Towards Realistic Data Generation for Real-World Super-Resolution](https://mlsanity.com/paper/2406.07255/) (2024)
- [Challenge-Aware RGBT Tracking](https://mlsanity.com/paper/2007.13143/) (2020)
- [Bi-directional Adapter for Multi-modal Tracking](https://mlsanity.com/paper/2312.10611/) (2023)
- [Lightweight Adaptive Feature De-drifting for Compressed Image Classification](https://mlsanity.com/paper/2401.01724/) (2024)
- [Cross-Modality Perturbation Synergy Attack for Person Re-identification](https://mlsanity.com/paper/2401.10090/) (2024)

## Abstract (arXiv)

Recently, visual prompt tuning is introduced to RGB-Thermal (RGB-T) tracking as a parameter-efficient finetuning (PEFT) method. However, these PEFT-based RGB-T tracking methods typically rely solely on spatial domain information as prompts for feature extraction. As a result, they often fail to achieve optimal performance by overlooking the crucial role of frequency-domain information in prompt learning. To address this issue, we propose an efficient Visual Fourier Prompt Tracking (named VFPTrack) method to learn modality-related prompts via Fast Fourier Transform (FFT). Our method consists of symmetric feature extraction encoder with shared parameters, visual fourier prompts, and Modality Fusion Prompt Generator that generates bidirectional interaction prompts through multi-modal feature fusion. Specifically, we first use a frozen feature extraction encoder to extract RGB and thermal infrared (TIR) modality features. Then, we combine the visual prompts in the spatial domain with the frequency domain prompts obtained from the FFT, which allows for the full extraction and understanding of modality features from different domain information. Finally, unlike previous fusion methods, the modality fusion prompt generation module we use combines features from different modalities to generate a fused modality prompt. This modality prompt is interacted with each individual modality to fully enable feature interaction across different modalities. Extensive experiments conducted on three popular RGB-T tracking benchmarks show that our method demonstrates outstanding performance.