# Low Latency Visual Inertial Odometry with On-Sensor Accelerated Optical Flow for Resource-Constrained UAVs

arXiv: https://arxiv.org/abs/2406.13345 · HTML: https://mlsanity.com/paper/2406.13345/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 48 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Builds on (most cited first, 15 of 15)

- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [On-Manifold Preintegration for Real-Time Visual-Inertial Odometry](https://mlsanity.com/paper/1512.02363/) (2015)
- [Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight](https://mlsanity.com/paper/2307.06100/) (2023)
- [Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight](https://mlsanity.com/paper/1712.00036/) (2017)
- [A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones](https://mlsanity.com/paper/1805.01831/) (2018)
- [Navion: A 2mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones](https://mlsanity.com/paper/1809.05780/) (2018)
- [Visual-Inertial Odometry of Aerial Robots](https://mlsanity.com/paper/1906.03289/) (2019)
- [Sniffy Bug: A Fully Autonomous Swarm of Gas-Seeking Nano Quadcopters in Cluttered Environments](https://mlsanity.com/paper/2107.05490/) (2021)
- [A Comparison of Modern General-Purpose Visual SLAM Approaches](https://mlsanity.com/paper/2107.07589/) (2021)
- [A Survey of FPGA-Based Robotic Computing](https://mlsanity.com/paper/2009.06034/) (2020)
- [Fusing Frame and Event Vision for High-speed Optical Flow for Edge Application](https://mlsanity.com/paper/2207.10720/) (2022)
- [A Fast and Accurate Optical Flow Camera for Resource-Constrained Edge Applications](https://mlsanity.com/paper/2305.13087/) (2023)

## Built on by (most cited first, 3 of 3)

- [LEVIO: Lightweight Embedded Visual Inertial Odometry for Resource-Constrained Devices](https://mlsanity.com/paper/2602.03294/) (2026)
- [PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation](https://mlsanity.com/paper/2603.11917/) (2026)
- [Efficient and Accurate Downfacing Visual Inertial Odometry](https://mlsanity.com/paper/2509.10021/) (2025)

## Abstract (arXiv)

Visual Inertial Odometry (VIO) is the task of estimating the movement trajectory of an agent from an onboard camera stream fused with additional Inertial Measurement Unit (IMU) measurements. A crucial subtask within VIO is the tracking of features, which can be achieved through Optical Flow (OF). As the calculation of OF is a resource-demanding task in terms of computational load and memory footprint, which needs to be executed at low latency, especially in robotic applications, OF estimation is today performed on powerful CPUs or GPUs. This restricts its use in a broad spectrum of applications where the deployment of such powerful, power-hungry processors is unfeasible due to constraints related to cost, size, and power consumption. On-sensor hardware acceleration is a promising approach to enable low latency VIO even on resource-constrained devices such as nano drones. This paper assesses the speed-up in a VIO sensor system exploiting a compact OF sensor consisting of a global shutter camera and an Application Specific Integrated Circuit (ASIC). By replacing the feature tracking logic of the VINS-Mono pipeline with data from this OF camera, we demonstrate a 49.4% reduction in latency and a 53.7% reduction of compute load of the VIO pipeline over the original VINS-Mono implementation, allowing VINS-Mono operation up to 50 FPS instead of 20 FPS on the quad-core ARM Cortex-A72 processor of a Raspberry Pi Compute Module 4.