# Validation of object detection in UAV-based images using synthetic data

arXiv: https://arxiv.org/abs/2201.06629 · HTML: https://mlsanity.com/paper/2201.06629/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 12 of 12)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking](https://mlsanity.com/paper/1804.00518/) (2018)
- [Meta-Sim: Learning to Generate Synthetic Datasets](https://mlsanity.com/paper/1904.11621/) (2019)
- [AU-AIR: A Multi-modal Unmanned Aerial Vehicle Dataset for Low Altitude Traffic Surveillance](https://mlsanity.com/paper/2001.11737/) (2020)
- [Okutama-Action: An Aerial View Video Dataset for Concurrent Human Action Detection](https://mlsanity.com/paper/1706.03038/) (2017)
- [Effective Use of Synthetic Data for Urban Scene Semantic Segmentation](https://mlsanity.com/paper/1807.06132/) (2018)
- [Delving into Robust Object Detection from Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach](https://mlsanity.com/paper/1908.03856/) (2019)
- [Bridging the Gap Between Computational Photography and Visual Recognition](https://mlsanity.com/paper/1901.09482/) (2019)
- [ME R-CNN: Multi-Expert R-CNN for Object Detection](https://mlsanity.com/paper/1704.01069/) (2017)

## Built on by (most cited first, 2 of 2)

- [Archangel: A Hybrid UAV-based Human Detection Benchmark with Position and Pose Metadata](https://mlsanity.com/paper/2209.00128/) (2022)
- [Can Synthetic UAV Benchmarks Be Trusted? Real-Synthetic Validation for UAV Model Selection](https://mlsanity.com/paper/2610.14647/) (2026)

## Abstract (arXiv)

Object detection is increasingly used onboard Unmanned Aerial Vehicles (UAV) for various applications; however, the machine learning (ML) models for UAV-based detection are often validated using data curated for tasks unrelated to the UAV application. This is a concern because training neural networks on large-scale benchmarks have shown excellent capability in generic object detection tasks, yet conventional training approaches can lead to large inference errors for UAV-based images. Such errors arise due to differences in imaging conditions between images from UAVs and images in training. To overcome this problem, we characterize boundary conditions of ML models, beyond which the models exhibit rapid degradation in detection accuracy. Our work is focused on understanding the impact of different UAV-based imaging conditions on detection performance by using synthetic data generated using a game engine. Properties of the game engine are exploited to populate the synthetic datasets with realistic and annotated images. Specifically, it enables the fine control of various parameters, such as camera position, view angle, illumination conditions, and object pose. Using the synthetic datasets, we analyze detection accuracy in different imaging conditions as a function of the above parameters. We use three well-known neural network models with different model complexity in our work. In our experiment, we observe and quantify the following: 1) how detection accuracy drops as the camera moves toward the nadir-view region; 2) how detection accuracy varies depending on different object poses, and 3) the degree to which the robustness of the models changes as illumination conditions vary.