# AutoPilot: Automating SoC Design Space Exploration for SWaP Constrained Autonomous UAVs

arXiv: https://arxiv.org/abs/2102.02988 · HTML: https://mlsanity.com/paper/2102.02988/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 71 references that are in the map.
- First posted on arXiv 5 Feb 2021.

## Builds on (most cited first, 17 of 17)

- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning](https://mlsanity.com/paper/1710.03937/) (2017)
- [A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones](https://mlsanity.com/paper/1805.01831/) (2018)
- [Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awareness](https://mlsanity.com/paper/1705.02550/) (2017)
- [Autonomous Navigation via Deep Reinforcement Learning for Resource Constraint Edge Nodes using Transfer Learning](https://mlsanity.com/paper/1910.05547/) (2019)
- [Fast, Autonomous Flight in GPS-Denied and Cluttered Environments](https://mlsanity.com/paper/1712.02052/) (2017)
- [Apollo: Transferable Architecture Exploration](https://mlsanity.com/paper/2102.01723/) (2021)
- [Navion: A 2mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones](https://mlsanity.com/paper/1809.05780/) (2018)
- [Air Learning: A Deep Reinforcement Learning Gym for Autonomous Aerial Robot Visual Navigation](https://mlsanity.com/paper/1906.00421/) (2019)
- [A Survey of FPGA-Based Robotic Computing](https://mlsanity.com/paper/2009.06034/) (2020)
- [Neuroflight: Next Generation Flight Control Firmware](https://mlsanity.com/paper/1901.06553/) (2019)
- [MAVBench: Micro Aerial Vehicle Benchmarking](https://mlsanity.com/paper/1905.06388/) (2019)
- [HiveMind: A Scalable and Serverless Coordination Control Platform for UAV Swarms](https://mlsanity.com/paper/2002.01419/) (2020)

## Abstract (arXiv)

Building domain-specific accelerators for autonomous unmanned aerial vehicles (UAVs) is challenging due to a lack of systematic methodology for designing onboard compute. Balancing a computing system for a UAV requires considering both the cyber (e.g., sensor rate, compute performance) and physical (e.g., payload weight) characteristics that affect overall performance. Iterating over the many component choices results in a combinatorial explosion of the number of possible combinations: from 10s of thousands to billions, depending on implementation details. Manually selecting combinations of these components is tedious and expensive. To navigate the cyber-physical design space efficiently, we introduce AutoPilot, a framework that automates full-system UAV co-design. AutoPilot uses Bayesian optimization to navigate a large design space and automatically select a combination of autonomy algorithm and hardware accelerator while considering the cross-product effect of other cyber and physical UAV components. We show that the AutoPilot methodology consistently outperforms general-purpose hardware selections like Xavier NX and Jetson TX2, as well as dedicated hardware accelerators built for autonomous UAVs, across a range of representative scenarios (three different UAV types and three deployment environments). Designs generated by AutoPilot increase the number of missions on average by up to 2.25x, 1.62x, and 1.43x for nano, micro, and mini-UAVs respectively over baselines. Our work demonstrates the need for holistic full-UAV co-design to achieve maximum overall UAV performance and the need for automated flows to simplify the design process for autonomous cyber-physical systems.