# Deep Learning for UAV-based Object Detection and Tracking: A Survey

arXiv: https://arxiv.org/abs/2110.12638 · HTML: https://mlsanity.com/paper/2110.12638/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 106 of its 243 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 30 of 106)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark](https://mlsanity.com/paper/1909.00133/) (2019)
- [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016)
- [The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking](https://mlsanity.com/paper/1804.00518/) (2018)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Arbitrary-Oriented Scene Text Detection via Rotation Proposals](https://mlsanity.com/paper/1703.01086/) (2017)
- [Drone-based Object Counting by Spatially Regularized Regional Proposal Network](https://mlsanity.com/paper/1707.05972/) (2017)
- [Detect to Track and Track to Detect](https://mlsanity.com/paper/1710.03958/) (2017)
- [Single-Shot Refinement Neural Network for Object Detection](https://mlsanity.com/paper/1711.06897/) (2017)
- [More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification](https://mlsanity.com/paper/2008.05457/) (2020)

## Built on by (most cited first, 30 of 33)

- [UIU-Net: U-Net in U-Net for Infrared Small Object Detection](https://mlsanity.com/paper/2212.00968/) (2022)
- [Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review](https://mlsanity.com/paper/2205.01380/) (2022)
- [WaterScenes: A Multi-Task 4D Radar-Camera Fusion Dataset and Benchmarks for Autonomous Driving on Water Surfaces](https://mlsanity.com/paper/2307.06505/) (2023)
- [Universal Adversarial Examples in Remote Sensing: Methodology and Benchmark](https://mlsanity.com/paper/2202.07054/) (2022)
- [A Comprehensive Survey for Hyperspectral Image Classification: The Evolution from Conventional to Transformers and Mamba Models](https://mlsanity.com/paper/2404.14955/) (2024)
- [Multiview Aerial Visual Recognition (MAVREC): Can Multi-view Improve Aerial Visual Perception?](https://mlsanity.com/paper/2312.04548/) (2023)
- [SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised Learning for Robust Infrared Small Target Detection](https://mlsanity.com/paper/2403.05416/) (2024)
- [Discrete Latent Perspective Learning for Segmentation and Detection](https://mlsanity.com/paper/2406.10475/) (2024)
- [BioDrone: A Bionic Drone-based Single Object Tracking Benchmark for Robust Vision](https://mlsanity.com/paper/2402.04519/) (2024)
- [Siamese Object Tracking for Unmanned Aerial Vehicle: A Review and Comprehensive Analysis](https://mlsanity.com/paper/2205.04281/) (2022)
- [Securing the Skies: A Comprehensive Survey on Anti-UAV Methods, Benchmarking, and Future Directions](https://mlsanity.com/paper/2504.11967/) (2025)
- [Archangel: A Hybrid UAV-based Human Detection Benchmark with Position and Pose Metadata](https://mlsanity.com/paper/2209.00128/) (2022)
- [Oriented object detection in optical remote sensing images using deep learning: a survey](https://mlsanity.com/paper/2302.10473/) (2023)
- [RemoteDet-Mamba: A Hybrid Mamba-CNN Network for Multi-modal Object Detection in Remote Sensing Images](https://mlsanity.com/paper/2410.13532/) (2024)
- [Efficient Object Detection in Optical Remote Sensing Imagery via Attention-based Feature Distillation](https://mlsanity.com/paper/2310.18676/) (2023)
- [Memory Maps for Video Object Detection and Tracking on UAVs](https://mlsanity.com/paper/2303.03508/) (2023)
- [UEMM-Air: Make Unmanned Aerial Vehicles Perform More Multi-modal Tasks](https://mlsanity.com/paper/2406.06230/) (2024)
- [Domain-invariant Progressive Knowledge Distillation for UAV-based Object Detection](https://mlsanity.com/paper/2408.11407/) (2024)
- [Lost in UNet: Improving Infrared Small Target Detection by Underappreciated Local Features](https://mlsanity.com/paper/2406.13445/) (2024)
- [Multimodal Collaboration Networks for Geospatial Vehicle Detection in Dense, Occluded, and Large-Scale Events](https://mlsanity.com/paper/2405.08251/) (2024)
- [SL-YOLO: A Stronger and Lighter Drone Target Detection Model](https://mlsanity.com/paper/2411.11477/) (2024)
- [YOLO-Drone:Airborne real-time detection of dense small objects from high-altitude perspective](https://mlsanity.com/paper/2304.06925/) (2023)
- [Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisons](https://mlsanity.com/paper/2311.07955/) (2023)
- [STCMOT: Spatio-Temporal Cohesion Learning for UAV-Based Multiple Object Tracking](https://mlsanity.com/paper/2409.11234/) (2024)
- [DREB-Net: Dual-stream Restoration Embedding Blur-feature Fusion Network for High-mobility UAV Object Detection](https://mlsanity.com/paper/2410.17822/) (2024)
- [A Review of Vision-Based Vehicle Detection for UAV-Based Traffic Monitoring: Experimental Insights and Future Directions](https://mlsanity.com/paper/2608.07571/) (2026)
- [Exploring the Effectiveness of Dataset Synthesis: An application of Apple Detection in Orchards](https://mlsanity.com/paper/2306.11763/) (2023)
- [STF: Spatio-Temporal Fusion Module for Improving Video Object Detection](https://mlsanity.com/paper/2402.10752/) (2024)
- [HierLight-YOLO: A Hierarchical and Lightweight Object Detection Network for UAV Photography](https://mlsanity.com/paper/2509.22365/) (2025)
- [DPTrack:Directional Kernel-Guided Prompt Learning for Robust Nighttime Aerial Tracking](https://mlsanity.com/paper/2510.15449/) (2025)

## Abstract (arXiv)

Owing to effective and flexible data acquisition, unmanned aerial vehicle (UAV) has recently become a hotspot across the fields of computer vision (CV) and remote sensing (RS). Inspired by recent success of deep learning (DL), many advanced object detection and tracking approaches have been widely applied to various UAV-related tasks, such as environmental monitoring, precision agriculture, traffic management. This paper provides a comprehensive survey on the research progress and prospects of DL-based UAV object detection and tracking methods. More specifically, we first outline the challenges, statistics of existing methods, and provide solutions from the perspectives of DL-based models in three research topics: object detection from the image, object detection from the video, and object tracking from the video. Open datasets related to UAV-dominated object detection and tracking are exhausted, and four benchmark datasets are employed for performance evaluation using some state-of-the-art methods. Finally, prospects and considerations for the future work are discussed and summarized. It is expected that this survey can facilitate those researchers who come from remote sensing field with an overview of DL-based UAV object detection and tracking methods, along with some thoughts on their further developments.