# Reinforcement Learning for Collision-free Flight Exploiting Deep Collision Encoding

arXiv: https://arxiv.org/abs/2402.03947 · HTML: https://mlsanity.com/paper/2402.03947/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 33 of its 48 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 30 of 33)

- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [BADGR: An Autonomous Self-Supervised Learning-Based Navigation System](https://mlsanity.com/paper/2002.05700/) (2020)
- [Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning](https://mlsanity.com/paper/2006.11751/) (2020)
- [Flightmare: A Flexible Quadrotor Simulator](https://mlsanity.com/paper/2009.00563/) (2020)
- [Deep Drone Racing: From Simulation to Reality with Domain Randomization](https://mlsanity.com/paper/1905.09727/) (2019)
- [Deep Drone Acrobatics](https://mlsanity.com/paper/2006.05768/) (2020)
- [Motion Planning Networks: Bridging the Gap Between Learning-based and Classical Motion Planners](https://mlsanity.com/paper/1907.06013/) (2019)
- [ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints](https://mlsanity.com/paper/2202.11271/) (2022)
- [Robot Motion Planning in Learned Latent Spaces](https://mlsanity.com/paper/1807.10366/) (2018)
- [Learning to Fly by Crashing](https://mlsanity.com/paper/1704.05588/) (2017)
- [Learning a State Representation and Navigation in Cluttered and Dynamic Environments](https://mlsanity.com/paper/2103.04351/) (2021)
- [Long-Range Indoor Navigation with PRM-RL](https://mlsanity.com/paper/1902.09458/) (2019)
- [AlphaPilot: Autonomous Drone Racing](https://mlsanity.com/paper/2005.12813/) (2020)
- [Locomotion Policy Guided Traversability Learning using Volumetric Representations of Complex Environments](https://mlsanity.com/paper/2203.15854/) (2022)
- [How Does It Feel? Self-Supervised Costmap Learning for Off-Road Vehicle Traversability](https://mlsanity.com/paper/2209.10788/) (2022)
- [LaND: Learning to Navigate from Disengagements](https://mlsanity.com/paper/2010.04689/) (2020)
- [Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing](https://mlsanity.com/paper/1810.06224/) (2018)
- [Learning-based 3D Occupancy Prediction for Autonomous Navigation in Occluded Environments](https://mlsanity.com/paper/2011.03981/) (2020)
- [CERBERUS: Autonomous Legged and Aerial Robotic Exploration in the Tunnel and Urban Circuits of the DARPA Subterranean Challenge](https://mlsanity.com/paper/2201.07067/) (2022)
- [NavRep: Unsupervised Representations for Reinforcement Learning of Robot Navigation in Dynamic Human Environments](https://mlsanity.com/paper/2012.04406/) (2020)
- [Team CERBERUS Wins the DARPA Subterranean Challenge: Technical Overview and Lessons Learned](https://mlsanity.com/paper/2207.04914/) (2022)
- [Visual Navigation Among Humans with Optimal Control as a Supervisor](https://mlsanity.com/paper/2003.09354/) (2020)
- [Motion Primitives-based Navigation Planning using Deep Collision Prediction](https://mlsanity.com/paper/2201.03254/) (2022)
- [Aerial Gym -- Isaac Gym Simulator for Aerial Robots](https://mlsanity.com/paper/2305.16510/) (2023)
- [Present and Future of SLAM in Extreme Underground Environments](https://mlsanity.com/paper/2208.01787/) (2022)
- [The Artificial Intelligence behind the winning entry to the 2019 AI Robotic Racing Competition](https://mlsanity.com/paper/2109.14985/) (2021)
- [Double Critic Deep Reinforcement Learning for Mapless 3D Navigation of Unmanned Aerial Vehicles](https://mlsanity.com/paper/2112.13724/) (2021)

## Built on by (most cited first, 13 of 13)

- [Flying on Point Clouds with Reinforcement Learning](https://mlsanity.com/paper/2503.00496/) (2025)
- [RAPID: Robust and Agile Planner Using Inverse Reinforcement Learning for Vision-Based Drone Navigation](https://mlsanity.com/paper/2502.02054/) (2025)
- [Reinforcement Learning for Active Perception in Autonomous Navigation](https://mlsanity.com/paper/2602.01266/) (2026)
- [Learning Long-Horizon Predictions for Quadrotor Dynamics](https://mlsanity.com/paper/2407.12964/) (2024)
- [Towards Task-Oriented Flying: Framework, Infrastructure, and Principles](https://mlsanity.com/paper/2504.15129/) (2025)
- [Synthetic vs. Real Training Data for Visual Navigation](https://mlsanity.com/paper/2509.11791/) (2025)
- [Non-Prehensile Aerial Manipulation using Model-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2407.00889/) (2024)
- [Zero-Shot UAV Navigation in Forests via Relightable 3D Gaussian Splatting](https://mlsanity.com/paper/2602.07101/) (2026)
- [Cross-Modal Reinforcement Learning for Navigation with Degraded Depth Measurements](https://mlsanity.com/paper/2603.22182/) (2026)
- [CMRL: Collision-Aware and Memory-Enhanced Reinforcement Learning for UAV Navigation in Multi-Scale Obstacle Environments](https://mlsanity.com/paper/2605.14810/) (2026)
- [A Heuristic Approach for Performance Tuning in RL-based Quadrotor Control via Reward Design and Termination Conditions](https://mlsanity.com/paper/2605.19166/) (2026)
- [Vision-Guided Outdoor Flight and Obstacle Evasion via Reinforcement Learning](https://mlsanity.com/paper/2605.24449/) (2026)
- [A 360-Degree Vision Dataset for Learning Yaw Control on GPS-Denied Micro-UAVs in Disaster-Response-Relevant Environments](https://mlsanity.com/paper/2608.19866/) (2026)

## Abstract (arXiv)

This work contributes a novel deep navigation policy that enables collision-free flight of aerial robots based on a modular approach exploiting deep collision encoding and reinforcement learning. The proposed solution builds upon a deep collision encoder that is trained on both simulated and real depth images using supervised learning such that it compresses the high-dimensional depth data to a low-dimensional latent space encoding collision information while accounting for the robot size. This compressed encoding is combined with an estimate of the robot's odometry and the desired target location to train a deep reinforcement learning navigation policy that offers low-latency computation and robust sim2real performance. A set of simulation and experimental studies in diverse environments are conducted and demonstrate the efficiency of the emerged behavior and its resilience in real-life deployments.