# Learning to Seek: Autonomous Source Seeking with Deep Reinforcement Learning Onboard a Nano Drone Microcontroller

arXiv: https://arxiv.org/abs/1909.11236 · HTML: https://mlsanity.com/paper/1909.11236/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2019.

## Built on by (most cited first, 4 of 4)

- [TCN Mapping Optimization for Ultra-Low Power Time-Series Edge Inference](https://mlsanity.com/paper/2203.12925/) (2022)
- [RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control](https://mlsanity.com/paper/2306.03530/) (2023)
- [DTMM: Deploying TinyML Models on Extremely Weak IoT Devices with Pruning](https://mlsanity.com/paper/2401.09068/) (2024)
- [Learning-based Bias Correction for Time Difference of Arrival Ultra-wideband Localization of Resource-constrained Mobile Robots](https://mlsanity.com/paper/2103.01885/) (2021)

## Abstract (arXiv)

We present fully autonomous source seeking onboard a highly constrained nano quadcopter, by contributing application-specific system and observation feature design to enable inference of a deep-RL policy onboard a nano quadcopter. Our deep-RL algorithm finds a high-performance solution to a challenging problem, even in presence of high noise levels and generalizes across real and simulation environments with different obstacle configurations. We verify our approach with simulation and in-field testing on a Bitcraze CrazyFlie using only the cheap and ubiquitous Cortex-M4 microcontroller unit. The results show that by end-to-end application-specific system design, our contribution consumes almost three times less additional power, as compared to competing learning-based navigation approach onboard a nano quadcopter. Thanks to our observation space, which we carefully design within the resource constraints, our solution achieves a 94% success rate in cluttered and randomized test environments, as compared to the previously achieved 80%. We also compare our strategy to a simple finite state machine (FSM), geared towards efficient exploration, and demonstrate that our policy is more robust and resilient at obstacle avoidance as well as up to 70% more efficient in source seeking. To this end, we contribute a cheap and lightweight end-to-end tiny robot learning (tinyRL) solution, running onboard a nano quadcopter, that proves to be robust and efficient in a challenging task using limited sensory input.