# On-device Self-supervised Learning of Visual Perception Tasks aboard Hardware-limited Nano-quadrotors

arXiv: https://arxiv.org/abs/2403.04071 · HTML: https://mlsanity.com/paper/2403.04071/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 6 Mar 2024.

## Builds on (most cited first, 19 of 19)

- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Real-World Robot Learning with Masked Visual Pre-training](https://mlsanity.com/paper/2210.03109/) (2022)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [On-Device Training Under 256KB Memory](https://mlsanity.com/paper/2206.15472/) (2022)
- [SelFlow: Self-Supervised Learning of Optical Flow](https://mlsanity.com/paper/1904.09117/) (2019)
- [TinyOL: TinyML with Online-Learning on Microcontrollers](https://mlsanity.com/paper/2103.08295/) (2021)
- [Learning to Fly by Crashing](https://mlsanity.com/paper/1704.05588/) (2017)
- [K for the Price of 1: Parameter-efficient Multi-task and Transfer Learning](https://mlsanity.com/paper/1810.10703/) (2018)
- [Fully Onboard AI-powered Human-Drone Pose Estimation on Ultra-low Power Autonomous Flying Nano-UAVs](https://mlsanity.com/paper/2103.10873/) (2021)
- [Geometric Consistency for Self-Supervised End-to-End Visual Odometry](https://mlsanity.com/paper/1804.03789/) (2018)
- [NanoFlowNet: Real-time Dense Optical Flow on a Nano Quadcopter](https://mlsanity.com/paper/2209.06918/) (2022)
- [Reduced Precision Floating-Point Optimization for Deep Neural Network On-Device Learning on MicroControllers](https://mlsanity.com/paper/2305.19167/) (2023)
- [Uncertainty-Aware Self-Supervised Learning of Spatial Perception Tasks](https://mlsanity.com/paper/2103.12007/) (2021)
- [RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control](https://mlsanity.com/paper/2306.03530/) (2023)
- [Sim-to-Real Vision-depth Fusion CNNs for Robust Pose Estimation Aboard Autonomous Nano-quadcopter](https://mlsanity.com/paper/2308.01833/) (2023)
- [Virtual Omnidirectional Perception for Downwash Prediction within a Team of Nano Multirotors Flying in Close Proximity](https://mlsanity.com/paper/2303.03898/) (2023)
- [RARA: Zero-shot Sim2Real Visual Navigation with Following Foreground Cues](https://mlsanity.com/paper/2201.02798/) (2022)

## Built on by (most cited first, 4 of 4)

- [LEVIO: Lightweight Embedded Visual Inertial Odometry for Resource-Constrained Devices](https://mlsanity.com/paper/2602.03294/) (2026)
- [Efficient and Accurate Downfacing Visual Inertial Odometry](https://mlsanity.com/paper/2509.10021/) (2025)
- [What changes after deployment? A survey on On-device Learning in TinyML](https://mlsanity.com/paper/2605.31226/) (2026)
- [TASTE: Throughput-Aware Batch Size Tuning for On-Device Edge Learning](https://mlsanity.com/paper/2609.07444/) (2026)

## Abstract (arXiv)

Sub-SI50gram nano-drones are gaining momentum in both academia and industry. Their most compelling applications rely on onboard deep learning models for perception despite severe hardware constraints (ie sub-SI100milliwatt processor). When deployed in unknown environments not represented in the training data, these models often underperform due to domain shift. To cope with this fundamental problem, we propose, for the first time, on-device learning aboard nano-drones, where the first part of the in-field mission is dedicated to self-supervised fine-tuning of a pre-trained convolutional neural network (CNN). Leveraging a real-world vision-based regression task, we thoroughly explore performance-cost trade-offs of the fine-tuning phase along three axes: textiti) dataset size (more data increases the regression performance but requires more memory and longer computation); textitii) methodologies (eg fine-tuning all model parameters vs. only a subset); and textitiii) self-supervision strategy. Our approach demonstrates an improvement in mean absolute error up to 30% compared to the pre-trained baseline, requiring only SI22second fine-tuning on an ultra-low-power GWT GAP9 System-on-Chip. Addressing the domain shift problem via on-device learning aboard nano-drones not only marks a novel result for hardware-limited robots but lays the ground for more general advancements for the entire robotics community.