# Event-driven Vision and Control for UAVs on a Neuromorphic Chip

arXiv: https://arxiv.org/abs/2108.03694 · HTML: https://mlsanity.com/paper/2108.03694/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 8 Aug 2021.

## Builds on (most cited first, 11 of 11)

- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Event-Based Angular Velocity Regression with Spiking Networks](https://mlsanity.com/paper/2003.02790/) (2020)
- [Real-time high speed motion prediction using fast aperture-robust event-driven visual flow](https://mlsanity.com/paper/1811.11135/) (2018)
- [A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones](https://mlsanity.com/paper/1805.01831/) (2018)
- [Nengo and low-power AI hardware for robust, embedded neurorobotics](https://mlsanity.com/paper/2007.10227/) (2020)
- [Towards Low-Latency High-Bandwidth Control of Quadrotors using Event Cameras](https://mlsanity.com/paper/1911.04553/) (2019)
- [Adaptive motor control and learning in a spiking neural network realised on a mixed-signal neuromorphic processor](https://mlsanity.com/paper/1810.10801/) (2018)
- [Closed-loop spiking control on a neuromorphic processor implemented on the iCub](https://mlsanity.com/paper/2009.09081/) (2020)

## Built on by (most cited first, 30 of 34)

- [Large Language Models for UAVs: Current State and Pathways to the Future](https://mlsanity.com/paper/2405.01745/) (2024)
- [Fully neuromorphic vision and control for autonomous drone flight](https://mlsanity.com/paper/2303.08778/) (2023)
- [Emergent Visual Sensors for Autonomous Vehicles](https://mlsanity.com/paper/2205.09383/) (2022)
- [Are High-Resolution Event Cameras Really Needed?](https://mlsanity.com/paper/2203.14672/) (2022)
- [EV-Catcher: High-Speed Object Catching Using Low-latency Event-based Neural Networks](https://mlsanity.com/paper/2304.07200/) (2023)
- [Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments](https://mlsanity.com/paper/2109.06452/) (2021)
- [Neuromorphic Camera Denoising using Graph Neural Network-driven Transformers](https://mlsanity.com/paper/2112.09685/) (2021)
- [Event-guided Low-light Video Semantic Segmentation](https://mlsanity.com/paper/2411.00639/) (2024)
- [Fusing Event-based Camera and Radar for SLAM Using Spiking Neural Networks with Continual STDP Learning](https://mlsanity.com/paper/2210.04236/) (2022)
- [Autonomous Driving with Spiking Neural Networks](https://mlsanity.com/paper/2405.19687/) (2024)
- [Applications of Spiking Neural Networks in Visual Place Recognition](https://mlsanity.com/paper/2311.13186/) (2023)
- [Marsellus: A Heterogeneous RISC-V AI-IoT End-Node SoC with 2-to-8b DNN Acceleration and 30%-Boost Adaptive Body Biasing](https://mlsanity.com/paper/2305.08415/) (2023)
- [Real-time event simulation with frame-based cameras](https://mlsanity.com/paper/2209.04634/) (2022)
- [Ensembles of Compact, Region-specific & Regularized Spiking Neural Networks for Scalable Place Recognition](https://mlsanity.com/paper/2209.08723/) (2022)
- [NYC-Event-VPR: A Large-Scale High-Resolution Event-Based Visual Place Recognition Dataset in Dense Urban Environments](https://mlsanity.com/paper/2410.21615/) (2024)
- [Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Network](https://mlsanity.com/paper/2412.16219/) (2024)
- [Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Application](https://mlsanity.com/paper/2503.22943/) (2025)
- [ColibriUAV: An Ultra-Fast, Energy-Efficient Neuromorphic Edge Processing UAV-Platform with Event-Based and Frame-Based Cameras](https://mlsanity.com/paper/2305.18371/) (2023)
- [HUE Dataset: High-Resolution Event and Frame Sequences for Low-Light Vision](https://mlsanity.com/paper/2410.19164/) (2024)
- [SEBVS: Synthetic Event-based Visual Servoing for Robot Navigation and Manipulation](https://mlsanity.com/paper/2508.17643/) (2025)
- [Density Invariant Contrast Maximization for Neuromorphic Earth Observations](https://mlsanity.com/paper/2304.14125/) (2023)
- [Neuromorphic Attitude Estimation and Control](https://mlsanity.com/paper/2411.13945/) (2024)
- [Design and implementation of a parsimonious neuromorphic PID for onboard altitude control for MAVs using neuromorphic processors](https://mlsanity.com/paper/2109.10199/) (2021)
- [Few-Shot Transfer Learning for Individualized Braking Intent Detection on Neuromorphic Hardware](https://mlsanity.com/paper/2408.03336/) (2024)
- [Low-Latency Scalable Streaming for Event-Based Vision](https://mlsanity.com/paper/2412.07889/) (2024)
- [Approximate Imitation Learning for Event-based Quadrotor Flight in Cluttered Environments](https://mlsanity.com/paper/2603.07578/) (2026)
- [Continuously Learning to Detect People on the Fly: A Bio-inspired Visual System for Drones](https://mlsanity.com/paper/2202.08023/) (2022)
- [Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networks](https://mlsanity.com/paper/2311.14265/) (2023)
- [A Neuromorphic Approach to Obstacle Avoidance in Robot Manipulation](https://mlsanity.com/paper/2404.05858/) (2024)
- [Robust Iterative Value Conversion: Deep Reinforcement Learning for Neurochip-driven Edge Robots](https://mlsanity.com/paper/2408.13018/) (2024)

## Abstract (arXiv)

Event-based vision sensors achieve up to three orders of magnitude better speed vs. power consumption trade off in high-speed control of UAVs compared to conventional image sensors. Event-based cameras produce a sparse stream of events that can be processed more efficiently and with a lower latency than images, enabling ultra-fast vision-driven control. Here, we explore how an event-based vision algorithm can be implemented as a spiking neuronal network on a neuromorphic chip and used in a drone controller. We show how seamless integration of event-based perception on chip leads to even faster control rates and lower latency. In addition, we demonstrate how online adaptation of the SNN controller can be realised using on-chip learning. Our spiking neuronal network on chip is the first example of a neuromorphic vision-based controller solving a high-speed UAV control task. The excellent scalability of processing in neuromorphic hardware opens the possibility to solve more challenging visual tasks in the future and integrate visual perception in fast control loops.