# Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking

arXiv: https://arxiv.org/abs/2504.09228 · HTML: https://mlsanity.com/paper/2504.09228/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 112 references that are in the map.
- First posted on arXiv 12 Apr 2025.

## Builds on (most cited first, 30 of 68)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Similarity-Preserving Knowledge Distillation](https://mlsanity.com/paper/1907.09682/) (2019)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [EVA-02: A Visual Representation for Neon Genesis](https://mlsanity.com/paper/2303.11331/) (2023)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks](https://mlsanity.com/paper/2004.05937/) (2020)
- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016)
- [Probabilistic Regression for Visual Tracking](https://mlsanity.com/paper/2003.12565/) (2020)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking](https://mlsanity.com/paper/1804.00518/) (2018)
- [Learning Target Candidate Association to Keep Track of What Not to Track](https://mlsanity.com/paper/2103.16556/) (2021)
- [Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition](https://mlsanity.com/paper/1905.04075/) (2019)
- [Backbone is All Your Need: A Simplified Architecture for Visual Object Tracking](https://mlsanity.com/paper/2203.05328/) (2022)
- [Transforming Model Prediction for Tracking](https://mlsanity.com/paper/2203.11192/) (2022)
- [HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network](https://mlsanity.com/paper/2203.14564/) (2022)

## Built on by (most cited first, 22 of 22)

- [PosterGen: Aesthetic-Aware Multi-Modal Paper-to-Poster Generation via Multi-Agent LLMs](https://mlsanity.com/paper/2508.17188/) (2025)
- [Vision-Based Anti Unmanned Aerial Technology: Opportunities and Challenges](https://mlsanity.com/paper/2507.10006/) (2025)
- [Instance-level Visual Active Tracking with Occlusion-Aware Planning](https://mlsanity.com/paper/2604.21453/) (2026)
- [AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World](https://mlsanity.com/paper/2606.29716/) (2026)
- [How Far are Modern Trackers from UAV-Anti-UAV? A Million-Scale Benchmark and New Baseline](https://mlsanity.com/paper/2512.07385/) (2025)
- [Decoupling Amplitude and Phase Attention in Frequency Domain for RGB-Event based Visual Object Tracking](https://mlsanity.com/paper/2601.01022/) (2026)
- [Layer-Guided UAV Tracking: Enhancing Efficiency and Occlusion Robustness](https://mlsanity.com/paper/2602.13636/) (2026)
- [SENTRY: SAM2-Enhanced Neighbor-Aware and Temporally Reasoned Memory for Visual Tracking](https://mlsanity.com/paper/2606.24449/) (2026)
- [Semantic-Aware Temporal Adaptation for UAV Anti-UAV Tracking](https://mlsanity.com/paper/2607.26511/) (2026)
- [T-SiamTPN: Temporal Siamese Transformer Pyramid Networks for Robust and Efficient UAV Tracking](https://mlsanity.com/paper/2509.12913/) (2025)
- [DPTrack:Directional Kernel-Guided Prompt Learning for Robust Nighttime Aerial Tracking](https://mlsanity.com/paper/2510.15449/) (2025)
- [CascadedViT: Cascaded Chunk-FeedForward and Cascaded Group Attention Vision Transformer](https://mlsanity.com/paper/2511.14111/) (2025)
- [Exploring Reliable Spatiotemporal Dependencies for Efficient Visual Tracking](https://mlsanity.com/paper/2601.09078/) (2026)
- [GOT-JEPA: Generic Object Tracking with Model Adaptation and Occlusion Handling using Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2602.14771/) (2026)
- [Geometric-Topological Perception and Motion Prior for Real-Time Satellite Video Object Tracking](https://mlsanity.com/paper/2603.07564/) (2026)
- [STATrack: A Target-Aware Fully Spiking Neural Network for Efficient UAV Tracking](https://mlsanity.com/paper/2603.27493/) (2026)
- [Dual-branch Distilled Transformer for Efficient Asymmetric UAV Tracking](https://mlsanity.com/paper/2605.28018/) (2026)
- [TAE: Target-aware enhancer for nighttime UAV tracking](https://mlsanity.com/paper/2605.29558/) (2026)
- [Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking](https://mlsanity.com/paper/2606.26455/) (2026)
- [SFDATrack: Generalized Source-Free Domain Adaptive Tracking Under Adverse Weather Conditions](https://mlsanity.com/paper/2607.00369/) (2026)
- [MaST: Motion-aware Sparse Pipeline for Lightweight Object Tracking](https://mlsanity.com/paper/2608.24365/) (2026)
- [SBMVTrack: Spike-Budgeted Multi-View Learning for Power-Efficient UAV Tracking](https://mlsanity.com/paper/2609.25503/) (2026)

## Abstract (arXiv)

Single-stream architectures using Vision Transformer (ViT) backbones show great potential for real-time UAV tracking recently. However, frequent occlusions from obstacles like buildings and trees expose a major drawback: these models often lack strategies to handle occlusions effectively. New methods are needed to enhance the occlusion resilience of single-stream ViT models in aerial tracking. In this work, we propose to learn Occlusion-Robust Representations (ORR) based on ViTs for UAV tracking by enforcing an invariance of the feature representation of a target with respect to random masking operations modeled by a spatial Cox process. Hopefully, this random masking approximately simulates target occlusions, thereby enabling us to learn ViTs that are robust to target occlusion for UAV tracking. This framework is termed ORTrack. Additionally, to facilitate real-time applications, we propose an Adaptive Feature-Based Knowledge Distillation (AFKD) method to create a more compact tracker, which adaptively mimics the behavior of the teacher model ORTrack according to the task's difficulty. This student model, dubbed ORTrack-D, retains much of ORTrack's performance while offering higher efficiency. Extensive experiments on multiple benchmarks validate the effectiveness of our method, demonstrating its state-of-the-art performance. Codes is available at https://github.com/wuyou3474/ORTrack.