# SGDViT: Saliency-Guided Dynamic Vision Transformer for UAV Tracking

arXiv: https://arxiv.org/abs/2303.04378 · HTML: https://mlsanity.com/paper/2303.04378/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 44 references that are in the map.
- First posted on arXiv 8 Mar 2023.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [Ocean: Object-aware Anchor-free Tracking](https://mlsanity.com/paper/2006.10721/) (2020)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines](https://mlsanity.com/paper/1911.06188/) (2019)
- [Deeper and Wider Siamese Networks for Real-Time Visual Tracking](https://mlsanity.com/paper/1901.01660/) (2019)
- [QuadTree Attention for Vision Transformers](https://mlsanity.com/paper/2201.02767/) (2022)
- [YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video](https://mlsanity.com/paper/1702.00824/) (2017)
- [Salient Object Detection: A Survey](https://mlsanity.com/paper/1411.5878/) (2014)
- [Learning the Model Update for Siamese Trackers](https://mlsanity.com/paper/1908.00855/) (2019)
- [Unsupervised Deep Tracking](https://mlsanity.com/paper/1904.01828/) (2019)
- [HiFT: Hierarchical Feature Transformer for Aerial Tracking](https://mlsanity.com/paper/2108.00202/) (2021)
- [AutoTrack: Towards High-Performance Visual Tracking for UAV with Automatic Spatio-Temporal Regularization](https://mlsanity.com/paper/2003.12949/) (2020)
- [Learning Aberrance Repressed Correlation Filters for Real-Time UAV Tracking](https://mlsanity.com/paper/1908.02231/) (2019)
- [Target-Aware Deep Tracking](https://mlsanity.com/paper/1904.01772/) (2019)
- [SiamAPN++: Siamese Attentional Aggregation Network for Real-Time UAV Tracking](https://mlsanity.com/paper/2106.08816/) (2021)
- [Scale Equivariance Improves Siamese Tracking](https://mlsanity.com/paper/2007.09115/) (2020)
- [Saliency-Associated Object Tracking](https://mlsanity.com/paper/2108.03637/) (2021)
- [Towards a Robust Aerial Cinematography Platform: Localizing and Tracking Moving Targets in Unstructured Environments](https://mlsanity.com/paper/1904.02319/) (2019)

## Built on by (most cited first, 20 of 20)

- [Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking](https://mlsanity.com/paper/2504.09228/) (2025)
- [SAM-DA: UAV Tracks Anything at Night with SAM-Powered Domain Adaptation](https://mlsanity.com/paper/2307.01024/) (2023)
- [MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking](https://mlsanity.com/paper/2503.17699/) (2025)
- [Progressive Representation Learning for Real-Time UAV Tracking](https://mlsanity.com/paper/2409.16652/) (2024)
- [DARTer: Dynamic Adaptive Representation Tracker for Nighttime UAV Tracking](https://mlsanity.com/paper/2505.00752/) (2025)
- [Benchmarking the Robustness of UAV Tracking Against Common Corruptions](https://mlsanity.com/paper/2403.11424/) (2024)
- [Enhancing Nighttime UAV Tracking with Light Distribution Suppression](https://mlsanity.com/paper/2409.16631/) (2024)
- [MambaNUT: Nighttime UAV Tracking via Mamba-based Adaptive Curriculum Learning](https://mlsanity.com/paper/2412.00626/) (2024)
- [Learning Motion Blur Robust Vision Transformers for Real-Time UAV Tracking](https://mlsanity.com/paper/2407.05383/) (2024)
- [Learning an Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking](https://mlsanity.com/paper/2412.20002/) (2024)
- [CGTrack: Cascade Gating Network with Hierarchical Feature Aggregation for UAV Tracking](https://mlsanity.com/paper/2505.05936/) (2025)
- [TrackingMiM: Efficient Mamba-in-Mamba Serialization for Real-time UAV Object Tracking](https://mlsanity.com/paper/2507.01535/) (2025)
- [Layer-Guided UAV Tracking: Enhancing Efficiency and Occlusion Robustness](https://mlsanity.com/paper/2602.13636/) (2026)
- [Target-aware Bidirectional Fusion Transformer for Aerial Object Tracking](https://mlsanity.com/paper/2503.09951/) (2025)
- [T-SiamTPN: Temporal Siamese Transformer Pyramid Networks for Robust and Efficient UAV Tracking](https://mlsanity.com/paper/2509.12913/) (2025)
- [DPTrack:Directional Kernel-Guided Prompt Learning for Robust Nighttime Aerial Tracking](https://mlsanity.com/paper/2510.15449/) (2025)
- [MATrack: Efficient Multiscale Adaptive Tracker for Real-Time Nighttime UAV Operations](https://mlsanity.com/paper/2510.21586/) (2025)
- [STATrack: A Target-Aware Fully Spiking Neural Network for Efficient UAV Tracking](https://mlsanity.com/paper/2603.27493/) (2026)
- [Dual-branch Distilled Transformer for Efficient Asymmetric UAV Tracking](https://mlsanity.com/paper/2605.28018/) (2026)
- [MaST: Motion-aware Sparse Pipeline for Lightweight Object Tracking](https://mlsanity.com/paper/2608.24365/) (2026)

## Abstract (arXiv)

Vision-based object tracking has boosted extensive autonomous applications for unmanned aerial vehicles (UAVs). However, the dynamic changes in flight maneuver and viewpoint encountered in UAV tracking pose significant difficulties, e.g. , aspect ratio change, and scale variation. The conventional cross-correlation operation, while commonly used, has limitations in effectively capturing perceptual similarity and incorporates extraneous background information. To mitigate these limitations, this work presents a novel saliency-guided dynamic vision Transformer (SGDViT) for UAV tracking. The proposed method designs a new task-specific object saliency mining network to refine the cross-correlation operation and effectively discriminate foreground and background information. Additionally, a saliency adaptation embedding operation dynamically generates tokens based on initial saliency, thereby reducing the computational complexity of the Transformer architecture. Finally, a lightweight saliency filtering Transformer further refines saliency information and increases the focus on appearance information. The efficacy and robustness of the proposed approach have been thoroughly assessed through experiments on three widely-used UAV tracking benchmarks and real-world scenarios, with results demonstrating its superiority. The source code and demo videos are available at https://github.com/vision4robotics/SGDViT.