# Self-Supervised Monocular Visual Drone Model Identification through Improved Occlusion Handling

arXiv: https://arxiv.org/abs/2504.21695 · HTML: https://mlsanity.com/paper/2504.21695/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 30 Apr 2025.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [Demonstrating Agile Flight from Pixels without State Estimation](https://mlsanity.com/paper/2406.12505/) (2024)
- [AlphaPilot: Autonomous Drone Racing](https://mlsanity.com/paper/2005.12813/) (2020)
- [Visual-Inertial Navigation: A Concise Review](https://mlsanity.com/paper/1906.02650/) (2019)
- [Learned Inertial Odometry for Autonomous Drone Racing](https://mlsanity.com/paper/2210.15287/) (2022)
- [The Artificial Intelligence behind the winning entry to the 2019 AI Robotic Racing Competition](https://mlsanity.com/paper/2109.14985/) (2021)
- [A Self-Supervised, Differentiable Kalman Filter for Uncertainty-Aware Visual-Inertial Odometry](https://mlsanity.com/paper/2203.07207/) (2022)
- [CUAHN-VIO: Content-and-Uncertainty-Aware Homography Network for Visual-Inertial Odometry](https://mlsanity.com/paper/2208.13935/) (2022)
- [CNN-based Ego-Motion Estimation for Fast MAV Maneuvers](https://mlsanity.com/paper/2101.01841/) (2021)
- [Race Against the Machine: a Fully-annotated, Open-design Dataset of Autonomous and Piloted High-speed Flight](https://mlsanity.com/paper/2311.02667/) (2023)

## Abstract (arXiv)

Ego-motion estimation is vital for drones when flying in GPS-denied environments. Vision-based methods struggle when flight speed increases and close-by objects lead to difficult visual conditions with considerable motion blur and large occlusions. To tackle this, vision is typically complemented by state estimation filters that combine a drone model with inertial measurements. However, these drone models are currently learned in a supervised manner with ground-truth data from external motion capture systems, limiting scalability to different environments and drones. In this work, we propose a self-supervised learning scheme to train a neural-network-based drone model using only onboard monocular video and flight controller data (IMU and motor feedback). We achieve this by first training a self-supervised relative pose estimation model, which then serves as a teacher for the drone model. To allow this to work at high speed close to obstacles, we propose an improved occlusion handling method for training self-supervised pose estimation models. Due to this method, the root mean squared error of resulting odometry estimates is reduced by an average of 15%. Moreover, the student neural drone model can be successfully obtained from the onboard data. It even becomes more accurate at higher speeds compared to its teacher, the self-supervised vision-based model. We demonstrate the value of the neural drone model by integrating it into a traditional filter-based VIO system (ROVIO), resulting in superior odometry accuracy on aggressive 3D racing trajectories near obstacles. Self-supervised learning of ego-motion estimation represents a significant step toward bridging the gap between flying in controlled, expensive lab environments and real-world drone applications. The fusion of vision and drone models will enable higher-speed flight and improve state estimation, on any drone in any environment.