# Leveraging Event Streams with Deep Reinforcement Learning for End-to-End UAV Tracking

arXiv: https://arxiv.org/abs/2410.14685 · HTML: https://mlsanity.com/paper/2410.14685/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [End-to-end Active Object Tracking via Reinforcement Learning](https://mlsanity.com/paper/1705.10561/) (2017)
- [Object Detection Using Sim2Real Domain Randomization for Robotic Applications](https://mlsanity.com/paper/2208.04171/) (2022)
- [Representation Learning for Event-based Visuomotor Policies](https://mlsanity.com/paper/2103.00806/) (2021)
- [Autonomous Navigation in Complex Environments with Deep Multimodal Fusion Network](https://mlsanity.com/paper/2007.15945/) (2020)
- [Enhancing Continuous Control of Mobile Robots for End-to-End Visual Active Tracking](https://mlsanity.com/paper/2009.13475/) (2020)
- [SARDO: An Automated Search-and-Rescue Drone-based Solution for Victims Localization](https://mlsanity.com/paper/2003.05819/) (2020)
- [D-VAT: End-to-End Visual Active Tracking for Micro Aerial Vehicles](https://mlsanity.com/paper/2308.16874/) (2023)
- [Exploration of Reinforcement Learning for Event Camera using Car-like Robots](https://mlsanity.com/paper/2004.00801/) (2020)
- [Towards Active Vision for Action Localization with Reactive Control and Predictive Learning](https://mlsanity.com/paper/2111.05448/) (2021)
- [Leader Follower Formation Control of Ground Vehicles Using Camshift Based Guidance](https://mlsanity.com/paper/1501.01364/) (2015)

## Built on by (most cited first, 1 of 1)

- [Human-Robot Navigation using Event-based Cameras and Reinforcement Learning](https://mlsanity.com/paper/2506.10790/) (2025)

## Abstract (arXiv)

In this paper, we present our proposed approach for active tracking to increase the autonomy of Unmanned Aerial Vehicles (UAVs) using event cameras, low-energy imaging sensors that offer significant advantages in speed and dynamic range. The proposed tracking controller is designed to respond to visual feedback from the mounted event sensor, adjusting the drone movements to follow the target. To leverage the full motion capabilities of a quadrotor and the unique properties of event sensors, we propose an end-to-end deep-reinforcement learning (DRL) framework that maps raw sensor data from event streams directly to control actions for the UAV. To learn an optimal policy under highly variable and challenging conditions, we opt for a simulation environment with domain randomization for effective transfer to real-world environments. We demonstrate the effectiveness of our approach through experiments in challenging scenarios, including fast-moving targets and changing lighting conditions, which result in improved generalization capabilities.