# Fully Onboard AI-powered Human-Drone Pose Estimation on Ultra-low Power Autonomous Flying Nano-UAVs

arXiv: https://arxiv.org/abs/2103.10873 · HTML: https://mlsanity.com/paper/2103.10873/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2021.

## Built on by (most cited first, 24 of 24)

- [Flexible and Fully Quantized Ultra-Lightweight TinyissimoYOLO for Ultra-Low-Power Edge Systems](https://mlsanity.com/paper/2307.05999/) (2023)
- [NanoFlowNet: Real-time Dense Optical Flow on a Nano Quadcopter](https://mlsanity.com/paper/2209.06918/) (2022)
- [Channel-wise Mixed-precision Assignment for DNN Inference on Constrained Edge Nodes](https://mlsanity.com/paper/2206.08852/) (2022)
- [GAP9Shield: A 150GOPS AI-capable Ultra-low Power Module for Vision and Ranging Applications on Nano-drones](https://mlsanity.com/paper/2407.13706/) (2024)
- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [On the Convergence of the Gradient Descent Method with Stochastic Fixed-point Rounding Errors under the Polyak-Lojasiewicz Inequality](https://mlsanity.com/paper/2301.09511/) (2023)
- [Adaptive Deep Learning for Efficient Visual Pose Estimation aboard Ultra-low-power Nano-drones](https://mlsanity.com/paper/2401.15236/) (2024)
- [On-device Self-supervised Learning of Visual Perception Tasks aboard Hardware-limited Nano-quadrotors](https://mlsanity.com/paper/2403.04071/) (2024)
- [Training on the Fly: On-device Self-supervised Learning aboard Nano-drones within 20 mW](https://mlsanity.com/paper/2408.03168/) (2024)
- [Deep Neural Network Architecture Search for Accurate Visual Pose Estimation aboard Nano-UAVs](https://mlsanity.com/paper/2303.01931/) (2023)
- [A Map-free Deep Learning-based Framework for Gate-to-Gate Monocular Visual Navigation aboard Miniaturized Aerial Vehicles](https://mlsanity.com/paper/2503.05251/) (2025)
- [A Heterogeneous In-Memory Computing Cluster For Flexible End-to-End Inference of Real-World Deep Neural Networks](https://mlsanity.com/paper/2201.01089/) (2022)
- [High-throughput Visual Nano-drone to Nano-drone Relative Localization using Onboard Fully Convolutional Networks](https://mlsanity.com/paper/2402.13756/) (2024)
- [Tiny-PULP-Dronets: Squeezing Neural Networks for Faster and Lighter Inference on Multi-Tasking Autonomous Nano-Drones](https://mlsanity.com/paper/2407.02405/) (2024)
- [Ultra-low Power Deep Learning-based Monocular Relative Localization Onboard Nano-quadrotors](https://mlsanity.com/paper/2303.01940/) (2023)
- [Flying Adversarial Patches: Manipulating the Behavior of Deep Learning-based Autonomous Multirotors](https://mlsanity.com/paper/2305.12859/) (2023)
- [Secure Deep Learning-based Distributed Intelligence on Pocket-sized Drones](https://mlsanity.com/paper/2307.01559/) (2023)
- [Kidnapping Deep Learning-based Multirotors using Optimized Flying Adversarial Patches](https://mlsanity.com/paper/2308.00344/) (2023)
- [Optimized Deployment of Deep Neural Networks for Visual Pose Estimation on Nano-drones](https://mlsanity.com/paper/2402.15273/) (2024)
- [Learning Quantized Continuous Controllers for Integer Hardware](https://mlsanity.com/paper/2511.07046/) (2025)
- [Training Lightweight CNNs for Human-Nanodrone Proximity Interaction from Small Datasets using Background Randomization](https://mlsanity.com/paper/2110.14491/) (2021)
- [Spiking Neural Network Enhanced Hand Gesture Recognition Using Low-Cost Single-photon Avalanche Diode Array](https://mlsanity.com/paper/2402.05441/) (2024)
- [AI and Vision based Autonomous Navigation of Nano-Drones in Partially-Known Environments](https://mlsanity.com/paper/2505.04972/) (2025)
- [Cyber Deception for Mission Surveillance via Hypergame-Theoretic Deep Reinforcement Learning](https://mlsanity.com/paper/2603.20981/) (2026)

## Abstract (arXiv)

Artificial intelligence-powered pocket-sized air robots have the potential to revolutionize the Internet-of-Things ecosystem, acting as autonomous, unobtrusive, and ubiquitous smart sensors. With a few cm2 form-factor, nano-sized unmanned aerial vehicles (UAVs) are the natural befit for indoor human-drone interaction missions, as the pose estimation task we address in this work. However, this scenario is challenged by the nano-UAVs' limited payload and computational power that severely relegates the onboard brain to the sub-100 mW microcontroller unit-class. Our work stands at the intersection of the novel parallel ultra-low-power (PULP) architectural paradigm and our general development methodology for deep neural network (DNN) visual pipelines, i.e., covering from perception to control. Addressing the DNN model design, from training and dataset augmentation to 8-bit quantization and deployment, we demonstrate how a PULP-based processor, aboard a nano-UAV, is sufficient for the real-time execution (up to 135 frame/s) of our novel DNN, called PULP-Frontnet. We showcase how, scaling our model's memory and computational requirement, we can significantly improve the onboard inference (top energy efficiency of 0.43 mJ/frame) with no compromise in the quality-of-result vs. a resource-unconstrained baseline (i.e., full-precision DNN). Field experiments demonstrate a closed-loop top-notch autonomous navigation capability, with a heavily resource-constrained 27-gram Crazyflie 2.1 nano-quadrotor. Compared against the control performance achieved using an ideal sensing setup, onboard relative pose inference yields excellent drone behavior in terms of median absolute errors, such as positional (onboard: 41 cm, ideal: 26 cm) and angular (onboard: 3.7circ, ideal: 4.1circ).