# Synthetic Data and Hierarchical Object Detection in Overhead Imagery

arXiv: https://arxiv.org/abs/2102.00103 · HTML: https://mlsanity.com/paper/2102.00103/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 29 Jan 2021.

## Builds on (most cited first, 22 of 22)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [xView: Objects in Context in Overhead Imagery](https://mlsanity.com/paper/1802.07856/) (2018)
- [Synthetic Data Augmentation using GAN for Improved Liver Lesion Classification](https://mlsanity.com/paper/1801.02385/) (2018)
- [GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks](https://mlsanity.com/paper/1810.10863/) (2018)
- [Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data](https://mlsanity.com/paper/1810.10093/) (2018)
- [The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classification](https://mlsanity.com/paper/1704.01664/) (2017)
- [Synthetic Medical Images from Dual Generative Adversarial Networks](https://mlsanity.com/paper/1709.01872/) (2017)
- [Pedestrian-Synthesis-GAN: Generating Pedestrian Data in Real Scene and Beyond](https://mlsanity.com/paper/1804.02047/) (2018)
- [Applying Domain Randomization to Synthetic Data for Object Category Detection](https://mlsanity.com/paper/1807.09834/) (2018)
- [Conditional Generative Adversarial Networks for Data Augmentation and Adaptation in Remotely Sensed Imagery](https://mlsanity.com/paper/1908.03809/) (2019)
- [Domain Adaptive Generation of Aircraft on Satellite Imagery via Simulated and Unsupervised Learning](https://mlsanity.com/paper/1806.03002/) (2018)

## Abstract (arXiv)

The performance of neural network models is often limited by the availability of big data sets. To treat this problem, we survey and develop novel synthetic data generation and augmentation techniques for enhancing low/zero-sample learning in satellite imagery. In addition to extending synthetic data generation approaches, we propose a hierarchical detection approach to improve the utility of synthetic training samples. We consider existing techniques for producing synthetic imagery--3D models and neural style transfer--as well as introducing our own adversarially trained reskinning network, the GAN-Reskinner, to blend 3D models. Additionally, we test the value of synthetic data in a two-stage, hierarchical detection/classification model of our own construction. To test the effectiveness of synthetic imagery, we employ it in the training of detection models and our two stage model, and evaluate the resulting models on real satellite images. All modalities of synthetic data are tested extensively on practical, geospatial analysis problems. Our experiments show that synthetic data developed using our approach can often enhance detection performance, particularly when combined with some real training images. When the only source of data is synthetic, our GAN-Reskinner often boosts performance over conventionally rendered 3D models and in all cases the hierarchical model outperforms the baseline end-to-end detection architecture.