# Scalable Modular Synthetic Data Generation for Advancing Aerial Autonomy

arXiv: https://arxiv.org/abs/2211.05335 · HTML: https://mlsanity.com/paper/2211.05335/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 10 Nov 2022.

## Builds on (most cited first, 26 of 26)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Tackling Climate Change with Machine Learning](https://mlsanity.com/paper/1906.05433/) (2019)
- [Meta-Sim: Learning to Generate Synthetic Datasets](https://mlsanity.com/paper/1904.11621/) (2019)
- [DriveGAN: Towards a Controllable High-Quality Neural Simulation](https://mlsanity.com/paper/2104.15060/) (2021)
- [Building Generalizable Agents with a Realistic and Rich 3D Environment](https://mlsanity.com/paper/1801.02209/) (2018)
- [Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing](https://mlsanity.com/paper/1810.08705/) (2018)
- [Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data](https://mlsanity.com/paper/1810.10093/) (2018)
- [Flightmare: A Flexible Quadrotor Simulator](https://mlsanity.com/paper/2009.00563/) (2020)
- [SceneGen: Learning to Generate Realistic Traffic Scenes](https://mlsanity.com/paper/2101.06541/) (2021)
- [Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generation](https://mlsanity.com/paper/2008.09092/) (2020)
- [Learning To Simulate](https://mlsanity.com/paper/1810.02513/) (2018)
- [CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data](https://mlsanity.com/paper/2112.09081/) (2021)
- [Segmentation-grounded Scene Graph Generation](https://mlsanity.com/paper/2104.14207/) (2021)
- [AirSim Drone Racing Lab](https://mlsanity.com/paper/2003.05654/) (2020)
- [A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?](https://mlsanity.com/paper/2108.11018/) (2021)
- [Optimizing Data Collection for Machine Learning](https://mlsanity.com/paper/2210.01234/) (2022)
- [Air Learning: A Deep Reinforcement Learning Gym for Autonomous Aerial Robot Visual Navigation](https://mlsanity.com/paper/1906.00421/) (2019)
- [Attribute Descent: Simulating Object-Centric Datasets on the Content Level and Beyond](https://mlsanity.com/paper/2202.14034/) (2022)
- [COMPASS: Contrastive Multimodal Pretraining for Autonomous Systems](https://mlsanity.com/paper/2203.15788/) (2022)
- [Learning to identify cracks on wind turbine blade surfaces using drone-based inspection images](https://mlsanity.com/paper/2207.11186/) (2022)

## Built on by (most cited first, 1 of 1)

- [What About the Data? A Mapping Study on Data Engineering for AI Systems](https://mlsanity.com/paper/2402.05156/) (2024)

## Abstract (arXiv)

One major barrier to advancing aerial autonomy has been collecting large-scale aerial datasets for training machine learning models. Due to costly and time-consuming real-world data collection through deploying drones, there has been an increasing shift towards using synthetic data for training models in drone applications. However, to increase widespread generalization and transferring models to real-world, increasing the diversity of simulation environments to train a model over all the varieties and augmenting the training data, has been proved to be essential. Current synthetic aerial data generation tools either lack data augmentation or rely heavily on manual workload or real samples for configuring and generating diverse realistic simulation scenes for data collection. These dependencies limit scalability of the data generation workflow. Accordingly, there is a major challenge in balancing generalizability and scalability in synthetic data generation. To address these gaps, we introduce a scalable Aerial Synthetic Data Augmentation (ASDA) framework tailored to aerial autonomy applications. ASDA extends a central data collection engine with two scriptable pipelines that automatically perform scene and data augmentations to generate diverse aerial datasets for different training tasks. ASDA improves data generation workflow efficiency by providing a unified prompt-based interface over integrated pipelines for flexible control. The procedural generative approach of our data augmentation is performant and adaptable to different simulation environments, training tasks and data collection needs. We demonstrate the effectiveness of our method in automatically generating diverse datasets and show its potential for downstream performance optimization.