# HiFT: Hierarchical Feature Transformer for Aerial Tracking

arXiv: https://arxiv.org/abs/2108.00202 · HTML: https://mlsanity.com/paper/2108.00202/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 55 references that are in the map.
- First posted on arXiv 31 Jul 2021.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016)
- [Probabilistic Regression for Visual Tracking](https://mlsanity.com/paper/2003.12565/) (2020)
- [End-to-End Dense Video Captioning with Masked Transformer](https://mlsanity.com/paper/1804.00819/) (2018)
- [Siamese Box Adaptive Network for Visual Tracking](https://mlsanity.com/paper/2003.06761/) (2020)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking](https://mlsanity.com/paper/1911.07241/) (2019)
- [YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video](https://mlsanity.com/paper/1702.00824/) (2017)
- [Unsupervised Deep Tracking](https://mlsanity.com/paper/1904.01828/) (2019)
- [Graph Attention Tracking](https://mlsanity.com/paper/2011.11204/) (2020)
- [Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking](https://mlsanity.com/paper/1608.03773/) (2016)
- [Learning Spatially Regularized Correlation Filters for Visual Tracking](https://mlsanity.com/paper/1608.05571/) (2016)
- [Learning Background-Aware Correlation Filters for Visual Tracking](https://mlsanity.com/paper/1703.04590/) (2017)

## Built on by (most cited first, 30 of 72)

- [AiATrack: Attention in Attention for Transformer Visual Tracking](https://mlsanity.com/paper/2207.09603/) (2022)
- [Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline](https://mlsanity.com/paper/2204.04120/) (2022)
- [TCTrack: Temporal Contexts for Aerial Tracking](https://mlsanity.com/paper/2203.01885/) (2022)
- [Unsupervised Domain Adaptation for Nighttime Aerial Tracking](https://mlsanity.com/paper/2203.10541/) (2022)
- [Large-Vocabulary 3D Diffusion Model with Transformer](https://mlsanity.com/paper/2309.07920/) (2023)
- [Separable Self and Mixed Attention Transformers for Efficient Object Tracking](https://mlsanity.com/paper/2309.03979/) (2023)
- [WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Tracking](https://mlsanity.com/paper/2201.07425/) (2022)
- [Towards Real-World Visual Tracking with Temporal Contexts](https://mlsanity.com/paper/2308.10330/) (2023)
- [Tracker Meets Night: A Transformer Enhancer for UAV Tracking](https://mlsanity.com/paper/2303.10951/) (2023)
- [Similarity-Guided Layer-Adaptive Vision Transformer for UAV Tracking](https://mlsanity.com/paper/2503.06625/) (2025)
- [Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking](https://mlsanity.com/paper/2504.09228/) (2025)
- [SGDViT: Saliency-Guided Dynamic Vision Transformer for UAV Tracking](https://mlsanity.com/paper/2303.04378/) (2023)
- [PTSEFormer: Progressive Temporal-Spatial Enhanced TransFormer Towards Video Object Detection](https://mlsanity.com/paper/2209.02242/) (2022)
- [SAM-DA: UAV Tracks Anything at Night with SAM-Powered Domain Adaptation](https://mlsanity.com/paper/2307.01024/) (2023)
- [LiteTrack: Layer Pruning with Asynchronous Feature Extraction for Lightweight and Efficient Visual Tracking](https://mlsanity.com/paper/2309.09249/) (2023)
- [DCPT: Darkness Clue-Prompted Tracking in Nighttime UAVs](https://mlsanity.com/paper/2309.10491/) (2023)
- [Siamese Transformer Pyramid Networks for Real-Time UAV Tracking](https://mlsanity.com/paper/2110.08822/) (2021)
- [HighlightNet: Highlighting Low-Light Potential Features for Real-Time UAV Tracking](https://mlsanity.com/paper/2208.06818/) (2022)
- [Siamese Object Tracking for Unmanned Aerial Vehicle: A Review and Comprehensive Analysis](https://mlsanity.com/paper/2205.04281/) (2022)
- [Tracking Small and Fast Moving Objects: A Benchmark](https://mlsanity.com/paper/2209.04284/) (2022)
- [Securing the Skies: A Comprehensive Survey on Anti-UAV Methods, Benchmarking, and Future Directions](https://mlsanity.com/paper/2504.11967/) (2025)
- [Transformers in Single Object Tracking: An Experimental Survey](https://mlsanity.com/paper/2302.11867/) (2023)
- [HOT: Higher-Order Dynamic Graph Representation Learning with Efficient Transformers](https://mlsanity.com/paper/2311.18526/) (2023)
- [Towards Discriminative Representations with Contrastive Instances for Real-Time UAV Tracking](https://mlsanity.com/paper/2308.11450/) (2023)
- [DiffTF++: 3D-aware Diffusion Transformer for Large-Vocabulary 3D Generation](https://mlsanity.com/paper/2405.08055/) (2024)
- [Progressive Representation Learning for Real-Time UAV Tracking](https://mlsanity.com/paper/2409.16652/) (2024)
- [Rank-Based Filter Pruning for Real-Time UAV Tracking](https://mlsanity.com/paper/2207.01768/) (2022)
- [VoxDet: Voxel Learning for Novel Instance Detection](https://mlsanity.com/paper/2305.17220/) (2023)
- [Distractor-aware Event-based Tracking](https://mlsanity.com/paper/2310.14194/) (2023)
- [CST Anti-UAV: A Thermal Infrared Benchmark for Tiny UAV Tracking in Complex Scenes](https://mlsanity.com/paper/2507.23473/) (2025)

## Abstract (arXiv)

Most existing Siamese-based tracking methods execute the classification and regression of the target object based on the similarity maps. However, they either employ a single map from the last convolutional layer which degrades the localization accuracy in complex scenarios or separately use multiple maps for decision making, introducing intractable computations for aerial mobile platforms. Thus, in this work, we propose an efficient and effective hierarchical feature transformer (HiFT) for aerial tracking. Hierarchical similarity maps generated by multi-level convolutional layers are fed into the feature transformer to achieve the interactive fusion of spatial (shallow layers) and semantics cues (deep layers). Consequently, not only the global contextual information can be raised, facilitating the target search, but also our end-to-end architecture with the transformer can efficiently learn the interdependencies among multi-level features, thereby discovering a tracking-tailored feature space with strong discriminability. Comprehensive evaluations on four aerial benchmarks have proven the effectiveness of HiFT. Real-world tests on the aerial platform have strongly validated its practicability with a real-time speed. Our code is available at https://github.com/vision4robotics/HiFT.