# MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking

arXiv: https://arxiv.org/abs/2503.17699 · HTML: https://mlsanity.com/paper/2503.17699/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 22 Mar 2025.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [Detection and Tracking Meet Drones Challenge](https://mlsanity.com/paper/2001.06303/) (2020)
- [ODTrack: Online Dense Temporal Token Learning for Visual Tracking](https://mlsanity.com/paper/2401.01686/) (2024)
- [Transforming Model Prediction for Tracking](https://mlsanity.com/paper/2203.11192/) (2022)
- [Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline](https://mlsanity.com/paper/2204.04120/) (2022)
- [Perceptual Generative Adversarial Networks for Small Object Detection](https://mlsanity.com/paper/1706.05274/) (2017)
- [HIPTrack: Visual Tracking with Historical Prompts](https://mlsanity.com/paper/2311.02072/) (2023)
- [Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance](https://mlsanity.com/paper/2403.05231/) (2024)
- [Learn to Match: Automatic Matching Network Design for Visual Tracking](https://mlsanity.com/paper/2108.00803/) (2021)
- [ARTrackV2: Prompting Autoregressive Tracker Where to Look and How to Describe](https://mlsanity.com/paper/2312.17133/) (2023)
- [Compact Transformer Tracker with Correlative Masked Modeling](https://mlsanity.com/paper/2301.10938/) (2023)
- [Material Based Object Tracking in Hyperspectral Videos: Benchmark and Algorithms](https://mlsanity.com/paper/1812.04179/) (2018)
- [Scale-aware Fast R-CNN for Pedestrian Detection](https://mlsanity.com/paper/1510.08160/) (2015)
- [ZoomTrack: Target-aware Non-uniform Resizing for Efficient Visual Tracking](https://mlsanity.com/paper/2310.10071/) (2023)
- [SGDViT: Saliency-Guided Dynamic Vision Transformer for UAV Tracking](https://mlsanity.com/paper/2303.04378/) (2023)
- [Beyond SOT: Tracking Multiple Generic Objects at Once](https://mlsanity.com/paper/2212.11920/) (2022)
- [DMSSN: Distilled Mixed Spectral-Spatial Network for Hyperspectral Salient Object Detection](https://mlsanity.com/paper/2404.00694/) (2024)

## Built on by (most cited first, 15 of 15)

- [Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music](https://mlsanity.com/paper/2604.10905/) (2026)
- [CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration](https://mlsanity.com/paper/2604.05689/) (2026)
- [Efficient Spike-driven Transformer for High-performance Drone-View Geo-Localization](https://mlsanity.com/paper/2512.19365/) (2025)
- [RAGTrack: Language-aware RGBT Tracking with Retrieval-Augmented Generation](https://mlsanity.com/paper/2603.03617/) (2026)
- [MSITrack: A Challenging Benchmark for Multispectral Single Object Tracking](https://mlsanity.com/paper/2510.06619/) (2025)
- [Decoupling Amplitude and Phase Attention in Frequency Domain for RGB-Event based Visual Object Tracking](https://mlsanity.com/paper/2601.01022/) (2026)
- [SENTRY: SAM2-Enhanced Neighbor-Aware and Temporally Reasoned Memory for Visual Tracking](https://mlsanity.com/paper/2606.24449/) (2026)
- [T-SiamTPN: Temporal Siamese Transformer Pyramid Networks for Robust and Efficient UAV Tracking](https://mlsanity.com/paper/2509.12913/) (2025)
- [MMOT: The First Challenging Benchmark for Drone-based Multispectral Multi-Object Tracking](https://mlsanity.com/paper/2510.12565/) (2025)
- [LER-YOLO: Reliability-Aware Expert Routing for Misaligned RGB-Infrared UAV Detection](https://mlsanity.com/paper/2605.20667/) (2026)
- [Towards UAV Detection in the Real World: A New Multispectral Dataset UAVNet-MS and a New Method](https://mlsanity.com/paper/2605.20963/) (2026)
- [Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking](https://mlsanity.com/paper/2606.26455/) (2026)
- [RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registration](https://mlsanity.com/paper/2606.30492/) (2026)
- [SpecTrack: Spectral Prompt Guided Adaptive Experts for Multispectral Object Tracking](https://mlsanity.com/paper/2607.05988/) (2026)
- [A Unified Benchmark and Modality-Adaptive Network for Day-and-Night Drone-View Geo-Localization](https://mlsanity.com/paper/2607.25778/) (2026)

## Abstract (arXiv)

UAV tracking faces significant challenges in real-world scenarios, such as small-size targets and occlusions, which limit the performance of RGB-based trackers. Multispectral images (MSI), which capture additional spectral information, offer a promising solution to these challenges. However, progress in this field has been hindered by the lack of relevant datasets. To address this gap, we introduce the first large-scale Multispectral UAV Single Object Tracking dataset (MUST), which includes 250 video sequences spanning diverse environments and challenges, providing a comprehensive data foundation for multispectral UAV tracking. We also propose a novel tracking framework, UNTrack, which encodes unified spectral, spatial, and temporal features from spectrum prompts, initial templates, and sequential searches. UNTrack employs an asymmetric transformer with a spectral background eliminate mechanism for optimal relationship modeling and an encoder that continuously updates the spectrum prompt to refine tracking, improving both accuracy and efficiency. Extensive experiments show that our proposed UNTrack outperforms state-of-the-art UAV trackers. We believe our dataset and framework will drive future research in this area. The dataset is available on https://github.com/q2479036243/MUST-Multispectral-UAV-Single-Object-Tracking.