# Architecture and evaluation protocol for transformer-based visual object tracking in UAV applications

arXiv: https://arxiv.org/abs/2603.03904 · HTML: https://mlsanity.com/paper/2603.03904/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 4 Mar 2026.

## Builds on (most cited first, 12 of 12)

- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://mlsanity.com/paper/2206.14651/) (2022)
- [Need for Speed: A Benchmark for Higher Frame Rate Object Tracking](https://mlsanity.com/paper/1703.05884/) (2017)
- [A Novel Performance Evaluation Methodology for Single-Target Trackers](https://mlsanity.com/paper/1503.01313/) (2015)
- [Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline](https://mlsanity.com/paper/2204.04120/) (2022)
- [MixFormerV2: Efficient Fully Transformer Tracking](https://mlsanity.com/paper/2305.15896/) (2023)
- [ChatTracker: Enhancing Visual Tracking Performance via Chatting with Multimodal Large Language Model](https://mlsanity.com/paper/2411.01756/) (2024)
- [Improving Visual Object Tracking through Visual Prompting](https://mlsanity.com/paper/2409.18901/) (2024)

## Abstract (arXiv)

Object tracking from Unmanned Aerial Vehicles (UAVs) is challenged by platform dynamics, camera motion, and limited onboard resources. Existing visual trackers either lack robustness in complex scenarios or are too computationally demanding for real-time embedded use. We propose an Modular Asynchronous Tracking Architecture (MATA) that combines a transformer-based tracker with an Extended Kalman Filter, integrating ego-motion compensation from sparse optical flow and an object trajectory model. We further introduce a hardware-independent, embedded oriented evaluation protocol and a new metric called Normalized time to Failure (NT2F) to quantify how long a tracker can sustain a tracking sequence without external help. Experiments on UAV benchmarks, including an augmented UAV123 dataset with synthetic occlusions, show consistent improvements in Success and NT2F metrics across multiple tracking processing frequency. A ROS 2 implementation on a Nvidia Jetson AGX Orin confirms that the evaluation protocol more closely matches real-time performance on embedded systems.