# NavRL: Learning Safe Flight in Dynamic Environments

arXiv: https://arxiv.org/abs/2409.15634 · HTML: https://mlsanity.com/paper/2409.15634/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 38 of its 56 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 30 of 38)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [ViNT: A Foundation Model for Visual Navigation](https://mlsanity.com/paper/2306.14846/) (2023)
- [FAST-LIO2: Fast Direct LiDAR-inertial Odometry](https://mlsanity.com/paper/2107.06829/) (2021)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [GNM: A General Navigation Model to Drive Any Robot](https://mlsanity.com/paper/2210.03370/) (2022)
- [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1805.01956/) (2018)
- [Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion](https://mlsanity.com/paper/2401.17583/) (2024)
- [EGO-Planner: An ESDF-free Gradient-based Local Planner for Quadrotors](https://mlsanity.com/paper/2008.08835/) (2020)
- [FUEL: Fast UAV Exploration using Incremental Frontier Structure and Hierarchical Planning](https://mlsanity.com/paper/2010.11561/) (2020)
- [ViPlanner: Visual Semantic Imperative Learning for Local Navigation](https://mlsanity.com/paper/2310.00982/) (2023)
- [Learning to Fly by Crashing](https://mlsanity.com/paper/1704.05588/) (2017)
- [Learning a State Representation and Navigation in Cluttered and Dynamic Environments](https://mlsanity.com/paper/2103.04351/) (2021)
- [OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control](https://mlsanity.com/paper/2309.12825/) (2023)
- [Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guarantees](https://mlsanity.com/paper/2201.08355/) (2022)
- [Memory-based Deep Reinforcement Learning for Obstacle Avoidance in UAV with Limited Environment Knowledge](https://mlsanity.com/paper/1811.03307/) (2018)
- [Provably Safe Reinforcement Learning via Action Projection using Reachability Analysis and Polynomial Zonotopes](https://mlsanity.com/paper/2210.10691/) (2022)
- [Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning](https://mlsanity.com/paper/1706.09829/) (2017)
- [Learning Perception-Aware Agile Flight in Cluttered Environments](https://mlsanity.com/paper/2210.01841/) (2022)
- [Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awareness](https://mlsanity.com/paper/1705.02550/) (2017)
- [Onboard dynamic-object detection and tracking for autonomous robot navigation with RGB-D camera](https://mlsanity.com/paper/2303.00132/) (2023)
- [Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flight](https://mlsanity.com/paper/2309.09865/) (2023)
- [Visual Navigation Among Humans with Optimal Control as a Supervisor](https://mlsanity.com/paper/2003.09354/) (2020)
- [Deep Neural Network for Real-Time Autonomous Indoor Navigation](https://mlsanity.com/paper/1511.04668/) (2015)
- [Vision-aided UAV navigation and dynamic obstacle avoidance using gradient-based B-spline trajectory optimization](https://mlsanity.com/paper/2209.07003/) (2022)

## Built on by (most cited first, 26 of 26)

- [GRaD-Nav++: Vision-Language Model Enabled Visual Drone Navigation with Gaussian Radiance Fields and Differentiable Dynamics](https://mlsanity.com/paper/2506.14009/) (2025)
- [AeroDuo: Aerial Duo for UAV-based Vision and Language Navigation](https://mlsanity.com/paper/2508.15232/) (2025)
- [Flying on Point Clouds with Reinforcement Learning](https://mlsanity.com/paper/2503.00496/) (2025)
- [LoGoPlanner: Localization Grounded Navigation Policy with Metric-aware Visual Geometry](https://mlsanity.com/paper/2512.19629/) (2025)
- [SIGN: Safety-Aware Image-Goal Navigation for Autonomous Drones via Reinforcement Learning](https://mlsanity.com/paper/2508.12394/) (2025)
- [CARE: Enhancing Safety of Visual Navigation through Collision Avoidance via Repulsive Estimation](https://mlsanity.com/paper/2506.03834/) (2025)
- [MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learning](https://mlsanity.com/paper/2510.03142/) (2025)
- [Mastering Diverse, Unknown, and Cluttered Tracks for Robust Vision-Based Drone Racing](https://mlsanity.com/paper/2512.09571/) (2025)
- [OpenFrontier: General Navigation with Visual-Language Grounded Frontiers](https://mlsanity.com/paper/2603.05377/) (2026)
- [AirDreamer: Generalist Drone Navigation with World Models](https://mlsanity.com/paper/2606.03252/) (2026)
- [End-to-End Humanoid Robot Safe and Comfortable Locomotion Policy](https://mlsanity.com/paper/2508.07611/) (2025)
- [Fly360: Omnidirectional Obstacle Avoidance within Drone View](https://mlsanity.com/paper/2603.06573/) (2026)
- [IR-SIM: A Lightweight Declarative Simulator for Navigation Learning and Benchmarking](https://mlsanity.com/paper/2606.08729/) (2026)
- [Towards Task-Oriented Flying: Framework, Infrastructure, and Principles](https://mlsanity.com/paper/2504.15129/) (2025)
- [High-Speed Vision-Based Flight in Clutter with Safety-Shielded Reinforcement Learning](https://mlsanity.com/paper/2602.08653/) (2026)
- [NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation](https://mlsanity.com/paper/2605.15559/) (2026)
- [AttentionSwarm: Reinforcement Learning with Attention Control Barier Function for Crazyflie Drones in Dynamic Environments](https://mlsanity.com/paper/2503.07376/) (2025)
- [Zero-Shot UAV Navigation in Forests via Relightable 3D Gaussian Splatting](https://mlsanity.com/paper/2602.07101/) (2026)
- [Decentralized End-to-End Multi-AAV Pursuit Using Predictive Spatio-Temporal Observation via Deep Reinforcement Learning](https://mlsanity.com/paper/2603.24238/) (2026)
- [Massive Parallel Deep Reinforcement Learning for Active SLAM](https://mlsanity.com/paper/2603.25834/) (2026)
- [AgenticDiffusion: Multi-View Reasoning with View-Conditioned Diffusion Planning for Vision-Based UAV Navigation](https://mlsanity.com/paper/2606.04111/) (2026)
- [FlashNav: Training Deployable Robot Navigation Policies in Seconds](https://mlsanity.com/paper/2606.15846/) (2026)
- [Anticipatory Risk-Guided Reinforcement Learning for Safe Flight Through Dynamic Clutter](https://mlsanity.com/paper/2607.23565/) (2026)
- [Learning-Based Motion Planning for Dynamic Environments: From Foundational Algorithms to Emerging Paradigms](https://mlsanity.com/paper/2608.00625/) (2026)
- [PILOT: Privileged Imitation Learning for End-to-End Motion Planning of Autonomous UAVs under Partial Observability](https://mlsanity.com/paper/2608.14082/) (2026)
- [SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Control](https://mlsanity.com/paper/2608.21175/) (2026)

## Abstract (arXiv)

Safe flight in dynamic environments requires unmanned aerial vehicles (UAVs) to make effective decisions when navigating cluttered spaces with moving obstacles. Traditional approaches often decompose decision-making into hierarchical modules for prediction and planning. Although these handcrafted systems can perform well in specific settings, they might fail if environmental conditions change and often require careful parameter tuning. Additionally, their solutions could be suboptimal due to the use of inaccurate mathematical model assumptions and simplifications aimed at achieving computational efficiency. To overcome these limitations, this paper introduces the NavRL framework, a deep reinforcement learning-based navigation method built on the Proximal Policy Optimization (PPO) algorithm. NavRL utilizes our carefully designed state and action representations, allowing the learned policy to make safe decisions in the presence of both static and dynamic obstacles, with zero-shot transfer from simulation to real-world flight. Furthermore, the proposed method adopts a simple but effective safety shield for the trained policy, inspired by the concept of velocity obstacles, to mitigate potential failures associated with the black-box nature of neural networks. To accelerate the convergence, we implement the training pipeline using NVIDIA Isaac Sim, enabling parallel training with thousands of quadcopters. Simulation and physical experiments show that our method ensures safe navigation in dynamic environments and results in the fewest collisions compared to benchmarks.