# A Map-free Deep Learning-based Framework for Gate-to-Gate Monocular Visual Navigation aboard Miniaturized Aerial Vehicles

arXiv: https://arxiv.org/abs/2503.05251 · HTML: https://mlsanity.com/paper/2503.05251/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 7 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs](https://mlsanity.com/paper/2008.07127/) (2020)
- [PULP-NN: Accelerating Quantized Neural Networks on Parallel Ultra-Low-Power RISC-V Processors](https://mlsanity.com/paper/1908.11263/) (2019)
- [AlphaPilot: Autonomous Drone Racing](https://mlsanity.com/paper/2005.12813/) (2020)
- [Fully Onboard AI-powered Human-Drone Pose Estimation on Ultra-low Power Autonomous Flying Nano-UAVs](https://mlsanity.com/paper/2103.10873/) (2021)
- [Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing](https://mlsanity.com/paper/1810.06224/) (2018)
- [Sniffy Bug: A Fully Autonomous Swarm of Gas-Seeking Nano Quadcopters in Cluttered Environments](https://mlsanity.com/paper/2107.05490/) (2021)
- [The Artificial Intelligence behind the winning entry to the 2019 AI Robotic Racing Competition](https://mlsanity.com/paper/2109.14985/) (2021)
- [PencilNet: Zero-Shot Sim-to-Real Transfer Learning for Robust Gate Perception in Autonomous Drone Racing](https://mlsanity.com/paper/2207.14131/) (2022)
- [Autonomous drone race: A computationally efficient vision-based navigation and control strategy](https://mlsanity.com/paper/1809.05958/) (2018)
- [A Sim-to-Real Deep Learning-based Framework for Autonomous Nano-drone Racing](https://mlsanity.com/paper/2312.08991/) (2023)
- [Distilling Tiny and Ultra-fast Deep Neural Networks for Autonomous Navigation on Nano-UAVs](https://mlsanity.com/paper/2407.12675/) (2024)
- [High-throughput Visual Nano-drone to Nano-drone Relative Localization using Onboard Fully Convolutional Networks](https://mlsanity.com/paper/2402.13756/) (2024)
- [Tiny-PULP-Dronets: Squeezing Neural Networks for Faster and Lighter Inference on Multi-Tasking Autonomous Nano-Drones](https://mlsanity.com/paper/2407.02405/) (2024)

## Built on by (most cited first, 3 of 3)

- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [HUNT: High-Speed UAV Navigation and Tracking in Unstructured Environments via Instantaneous Relative Frames](https://mlsanity.com/paper/2509.19452/) (2025)
- [MR2-ByteTrack: CNN and Transformer-based Video Object Detection for AI-augmented Embedded Vision Sensor Nodes](https://mlsanity.com/paper/2605.15423/) (2026)

## Abstract (arXiv)

Palm-sized autonomous nano-drones, i.e., sub-50g in weight, recently entered the drone racing scenario, where they are tasked to avoid obstacles and navigate as fast as possible through gates. However, in contrast with their bigger counterparts, i.e., kg-scale drones, nano-drones expose three orders of magnitude less onboard memory and compute power, demanding more efficient and lightweight vision-based pipelines to win the race. This work presents a map-free vision-based (using only a monocular camera) autonomous nano-drone that combines a real-time deep learning gate detection front-end with a classic yet elegant and effective visual servoing control back-end, only relying on onboard resources. Starting from two state-of-the-art tiny deep learning models, we adapt them for our specific task, and after a mixed simulator-real-world training, we integrate and deploy them aboard our nano-drone. Our best-performing pipeline costs of only 24M multiply-accumulate operations per frame, resulting in a closed-loop control performance of 30 Hz, while achieving a gate detection root mean square error of 1.4 pixels, on our ~20k real-world image dataset. In-field experiments highlight the capability of our nano-drone to successfully navigate through 15 gates in 4 min, never crashing and covering a total travel distance of ~100m, with a peak flight speed of 1.9 m/s. Finally, to stress the generalization capability of our system, we also test it in a never-seen-before environment, where it navigates through gates for more than 4 min.