# Siamese Transformer Pyramid Networks for Real-Time UAV Tracking

arXiv: https://arxiv.org/abs/2110.08822 · HTML: https://mlsanity.com/paper/2110.08822/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 53 references that are in the map.
- First posted on arXiv 17 Oct 2021.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018)
- [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018)
- [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016)
- [Ocean: Object-aware Anchor-free Tracking](https://mlsanity.com/paper/2006.10721/) (2020)
- [Siamese Box Adaptive Network for Visual Tracking](https://mlsanity.com/paper/2003.06761/) (2020)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines](https://mlsanity.com/paper/1911.06188/) (2019)
- [SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking](https://mlsanity.com/paper/1911.07241/) (2019)
- [Deformable Siamese Attention Networks for Visual Object Tracking](https://mlsanity.com/paper/2004.06711/) (2020)
- [Learning to Track at 100 FPS with Deep Regression Networks](https://mlsanity.com/paper/1604.01802/) (2016)
- [HiFT: Hierarchical Feature Transformer for Aerial Tracking](https://mlsanity.com/paper/2108.00202/) (2021)
- [D3S -- A Discriminative Single Shot Segmentation Tracker](https://mlsanity.com/paper/1911.08862/) (2019)
- [Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking](https://mlsanity.com/paper/1608.03773/) (2016)
- [Learning Background-Aware Correlation Filters for Visual Tracking](https://mlsanity.com/paper/1703.04590/) (2017)
- [Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking](https://mlsanity.com/paper/1803.08679/) (2018)

## Built on by (most cited first, 15 of 15)

- [AiATrack: Attention in Attention for Transformer Visual Tracking](https://mlsanity.com/paper/2207.09603/) (2022)
- [Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline](https://mlsanity.com/paper/2204.04120/) (2022)
- [Vision-based Anti-UAV Detection and Tracking](https://mlsanity.com/paper/2205.10851/) (2022)
- [SGDViT: Saliency-Guided Dynamic Vision Transformer for UAV Tracking](https://mlsanity.com/paper/2303.04378/) (2023)
- [In Pursuit of Many: A Review of Modern Multiple Object Tracking Systems](https://mlsanity.com/paper/2209.04796/) (2022)
- [Transformers in Single Object Tracking: An Experimental Survey](https://mlsanity.com/paper/2302.11867/) (2023)
- [In Defense and Revival of Bayesian Filtering for Thermal Infrared Object Tracking](https://mlsanity.com/paper/2402.17098/) (2024)
- [Multi-step Temporal Modeling for UAV Tracking](https://mlsanity.com/paper/2403.04363/) (2024)
- [Recent Advances in Transformer and Large Language Models for UAV Applications](https://mlsanity.com/paper/2508.11834/) (2025)
- [FARTrack: Fast Autoregressive Visual Tracking with High Performance](https://mlsanity.com/paper/2602.03214/) (2026)
- [CGTrack: Cascade Gating Network with Hierarchical Feature Aggregation for UAV Tracking](https://mlsanity.com/paper/2505.05936/) (2025)
- [Performance Evaluation of Deep Learning-based Quadrotor UAV Detection and Tracking Methods](https://mlsanity.com/paper/2410.00285/) (2024)
- [Target-aware Bidirectional Fusion Transformer for Aerial Object Tracking](https://mlsanity.com/paper/2503.09951/) (2025)
- [T-SiamTPN: Temporal Siamese Transformer Pyramid Networks for Robust and Efficient UAV Tracking](https://mlsanity.com/paper/2509.12913/) (2025)
- [Banshee: Target Switch Attacks on Gimbal-Stabilized Visual Tracking Systems via Acoustic Injection](https://mlsanity.com/paper/2607.09930/) (2026)

## Abstract (arXiv)

Recent object tracking methods depend upon deep networks or convoluted architectures. Most of those trackers can hardly meet real-time processing requirements on mobile platforms with limited computing resources. In this work, we introduce the Siamese Transformer Pyramid Network (SiamTPN), which inherits the advantages from both CNN and Transformer architectures. Specifically, we exploit the inherent feature pyramid of a lightweight network (ShuffleNetV2) and reinforce it with a Transformer to construct a robust target-specific appearance model. A centralized architecture with lateral cross attention is developed for building augmented high-level feature maps. To avoid the computation and memory intensity while fusing pyramid representations with the Transformer, we further introduce the pooling attention module, which significantly reduces memory and time complexity while improving the robustness. Comprehensive experiments on both aerial and prevalent tracking benchmarks achieve competitive results while operating at high speed, demonstrating the effectiveness of SiamTPN. Moreover, our fastest variant tracker operates over 30 Hz on a single CPU-core and obtaining an AUC score of 58.1% on the LaSOT dataset. Source codes are available at https://github.com/RISCNYUAD/SiamTPNTracker