Leveraging Event Streams with Deep Reinforcement Learning for End-to-End UAV Tracking
On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control
Abstract (arXiv)
In this paper, we present our proposed approach for active tracking to increase the autonomy of Unmanned Aerial Vehicles (UAVs) using event cameras, low-energy imaging sensors that offer significant advantages in speed and dynamic range. The proposed tracking controller is designed to respond to visual feedback from the mounted event sensor, adjusting the drone movements to follow the target. To leverage the full motion capabilities of a quadrotor and the unique properties of event sensors, we propose an end-to-end deep-reinforcement learning (DRL) framework that maps raw sensor data from event streams directly to control actions for the UAV. To learn an optimal policy under highly variable and challenging conditions, we opt for a simulation environment with domain randomization for effective transfer to real-world environments. We demonstrate the effectiveness of our approach through experiments in challenging scenarios, including fast-moving targets and changing lighting conditions, which result in improved generalization capabilities.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 33 references that are in the map. First posted on arXiv 3 Oct 2024.
Builds on
18 of 33 references.
References · 18
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
- 2015SSD: Single Shot MultiBox Detectorcited by 3,130
- 2019Event-based Vision: A Surveycited by 1,016
- 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
- 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
- 2024YOLOv9: Learning What You Want to Learn Using Programmable Gradient Informationcited by 425
- 2021DSEC: A Stereo Event Camera Dataset for Driving Scenarioscited by 303
- 2017End-to-end Active Object Tracking via Reinforcement Learningcited by 28
- 2022Object Detection Using Sim2Real Domain Randomization for Robotic Applicationscited by 25
- 2021Representation Learning for Event-based Visuomotor Policiescited by 20
- 2020Autonomous Navigation in Complex Environments with Deep Multimodal Fusion Networkcited by 16
6 more
- 2020Enhancing Continuous Control of Mobile Robots for End-to-End Visual Active Trackingcited by 11
- 2020SARDO: An Automated Search-and-Rescue Drone-based Solution for Victims Localizationcited by 9
- 2023D-VAT: End-to-End Visual Active Tracking for Micro Aerial Vehiclescited by 6
- 2020Exploration of Reinforcement Learning for Event Camera using Car-like Robotscited by 5
- 2021Towards Active Vision for Action Localization with Reactive Control and Predictive Learningcited by 5
- 2015Leader Follower Formation Control of Ground Vehicles Using Camshift Based Guidancecited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Open-World Drone Active Tracking with Goal-Centered Rewards
- 2025AgilePilot: DRL-Based Drone Agent for Real-Time Motion Planning in Dynamic Environments by Leveraging Object Detection
- 2024Tangled Program Graphs as an alternative to DRL-based control algorithms for UAVs
- 2025Deep Reinforcement Learning Enabled Persistent Surveillance with Energy-Aware UAV-UGV Systems for Disaster Management Applications
- 2024Goal-Oriented UAV Communication Design and Optimization for Target Tracking: A MachineLearning Approach
- 2024DRAL: Deep Reinforcement Adaptive Learning for Multi-UAVs Navigation in Unknown Indoor Environment
- 2024Dual Agent Learning Based Aerial Trajectory Tracking
- 2025Towards Task-Oriented Flying: Framework, Infrastructure, and Principles