# Unmanned Aerial Vehicle Visual Detection and Tracking using Deep Neural Networks: A Performance Benchmark

arXiv: https://arxiv.org/abs/2103.13933 · HTML: https://mlsanity.com/paper/2103.13933/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 25 Mar 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [R-FCN: Object Detection via Region-based Fully Convolutional Networks](https://mlsanity.com/paper/1605.06409/) (2016)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks](https://mlsanity.com/paper/1312.6229/) (2013)
- [Deep Learning in Video Multi-Object Tracking: A Survey](https://mlsanity.com/paper/1907.12740/) (2019)
- [UG2+ Track 2: A Collective Benchmark Effort for Evaluating and Advancing Image Understanding in Poor Visibility Environments](https://mlsanity.com/paper/1904.04474/) (2019)
- [Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor](https://mlsanity.com/paper/1504.02340/) (2015)
- [Anti-UAV: A Large Multi-Modal Benchmark for UAV Tracking](https://mlsanity.com/paper/2101.08466/) (2021)
- [Flying Objects Detection from a Single Moving Camera](https://mlsanity.com/paper/1411.7715/) (2014)
- [Empirical Study of Drone Sound Detection in Real-Life Environment with Deep Neural Networks](https://mlsanity.com/paper/1701.05779/) (2017)
- [Temporal Dynamic Appearance Modeling for Online Multi-Person Tracking](https://mlsanity.com/paper/1510.02906/) (2015)

## Built on by (most cited first, 25 of 25)

- [Vision-based Anti-UAV Detection and Tracking](https://mlsanity.com/paper/2205.10851/) (2022)
- [Evidential Detection and Tracking Collaboration: New Problem, Benchmark and Algorithm for Robust Anti-UAV System](https://mlsanity.com/paper/2306.15767/) (2023)
- [Global-Local MAV Detection under Challenging Conditions based on Appearance and Motion](https://mlsanity.com/paper/2312.11008/) (2023)
- [SynDroneVision: A Synthetic Dataset for Image-Based Drone Detection](https://mlsanity.com/paper/2411.05633/) (2024)
- [Investigation of UAV Detection in Images with Complex Backgrounds and Rainy Artifacts](https://mlsanity.com/paper/2305.16450/) (2023)
- [Domain Adaptive Detection of MAVs: A Benchmark and Noise Suppression Network](https://mlsanity.com/paper/2403.16669/) (2024)
- [Strong Baseline: Multi-UAV Tracking via YOLOv12 with BoT-SORT-ReID](https://mlsanity.com/paper/2503.17237/) (2025)
- [Vision-Based Anti Unmanned Aerial Technology: Opportunities and Challenges](https://mlsanity.com/paper/2507.10006/) (2025)
- [UAV-ReID: A Benchmark on Unmanned Aerial Vehicle Re-identification in Video Imagery](https://mlsanity.com/paper/2104.06219/) (2021)
- [Real-Time Detection for Small UAVs: Combining YOLO and Multi-frame Motion Analysis](https://mlsanity.com/paper/2411.02582/) (2024)
- [Fusion Flow-enhanced Graph Pooling Residual Networks for Unmanned Aerial Vehicles Surveillance in Day and Night Dual Visions](https://mlsanity.com/paper/2407.12647/) (2024)
- [Motion-guided small MAV detection in complex and non-planar scenes](https://mlsanity.com/paper/2410.10527/) (2024)
- [Unsupervised UAV 3D Trajectories Estimation with Sparse Point Clouds](https://mlsanity.com/paper/2412.12716/) (2024)
- [YOLOMG: Vision-based Drone-to-Drone Detection with Appearance and Pixel-Level Motion Fusion](https://mlsanity.com/paper/2503.07115/) (2025)
- [GL-DT: Multi-UAV Detection and Tracking with Global-Local Integration](https://mlsanity.com/paper/2510.09092/) (2025)
- [Sequence Models for Drone vs Bird Classification](https://mlsanity.com/paper/2207.10409/) (2022)
- [Drone Detection using Deep Neural Networks Trained on Pure Synthetic Data](https://mlsanity.com/paper/2411.09077/) (2024)
- [Event-Only Drone Trajectory Forecasting with RPM-Modulated Kalman Filtering](https://mlsanity.com/paper/2603.01997/) (2026)
- [UAV-DETR: DETR for Anti-Drone Target Detection](https://mlsanity.com/paper/2603.22841/) (2026)
- [DroBoost: An Intelligent Score and Model Boosting Method for Drone Detection](https://mlsanity.com/paper/2407.00830/) (2024)
- [Separating Drone Point Clouds From Complex Backgrounds by Cluster Filter -- Technical Report for CVPR 2024 UG2 Challenge](https://mlsanity.com/paper/2412.16947/) (2024)
- [LASFNet: A Lightweight Attention-Guided Self-Modulation Feature Fusion Network for Multimodal Object Detection](https://mlsanity.com/paper/2506.21018/) (2025)
- [DMTrack: Deformable State-Space Modeling for UAV Multi-Object Tracking with Kalman Fusion and Uncertainty-Aware Association](https://mlsanity.com/paper/2510.17860/) (2025)
- [LAA3D: A Benchmark of Detecting and Tracking Low-Altitude Aircraft in 3D Space](https://mlsanity.com/paper/2511.19057/) (2025)
- [SimD3: A Synthetic drone Dataset with Payload and Bird Distractor Modeling for Robust Detection](https://mlsanity.com/paper/2601.14742/) (2026)

## Abstract (arXiv)

Unmanned Aerial Vehicles (UAV) can pose a major risk for aviation safety, due to both negligent and malicious use. For this reason, the automated detection and tracking of UAV is a fundamental task in aerial security systems. Common technologies for UAV detection include visible-band and thermal infrared imaging, radio frequency and radar. Recent advances in deep neural networks (DNNs) for image-based object detection open the possibility to use visual information for this detection and tracking task. Furthermore, these detection architectures can be implemented as backbones for visual tracking systems, thereby enabling persistent tracking of UAV incursions. To date, no comprehensive performance benchmark exists that applies DNNs to visible-band imagery for UAV detection and tracking. To this end, three datasets with varied environmental conditions for UAV detection and tracking, comprising a total of 241 videos (331,486 images), are assessed using four detection architectures and three tracking frameworks. The best performing detector architecture obtains an mAP of 98.6% and the best performing tracking framework obtains a MOTA of 96.3%. Cross-modality evaluation is carried out between visible and infrared spectrums, achieving a maximal 82.8% mAP on visible images when training in the infrared modality. These results provide the first public multi-approach benchmark for state-of-the-art deep learning-based methods and give insight into which detection and tracking architectures are effective in the UAV domain.