# Anticipatory Risk-Guided Reinforcement Learning for Safe Flight Through Dynamic Clutter

arXiv: https://arxiv.org/abs/2607.23565 · HTML: https://mlsanity.com/paper/2607.23565/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 24 references that are in the map.
- First posted on arXiv 26 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [Learning a State Representation and Navigation in Cluttered and Dynamic Environments](https://mlsanity.com/paper/2103.04351/) (2021)
- [DRL-VO: Learning to Navigate Through Crowded Dynamic Scenes Using Velocity Obstacles](https://mlsanity.com/paper/2301.06512/) (2023)
- [OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control](https://mlsanity.com/paper/2309.12825/) (2023)
- [NavRL: Learning Safe Flight in Dynamic Environments](https://mlsanity.com/paper/2409.15634/) (2024)
- [DPMPC-Planner: A real-time UAV trajectory planning framework for complex static environments with dynamic obstacles](https://mlsanity.com/paper/2109.07024/) (2021)
- [Learning Speed Adaptation for Flight in Clutter](https://mlsanity.com/paper/2403.04586/) (2024)
- [Robust Vision-based Obstacle Avoidance for Micro Aerial Vehicles in Dynamic Environments](https://mlsanity.com/paper/2002.04920/) (2020)
- [Deep-PANTHER: Learning-Based Perception-Aware Trajectory Planner in Dynamic Environments](https://mlsanity.com/paper/2209.01268/) (2022)
- [Vision-aided UAV navigation and dynamic obstacle avoidance using gradient-based B-spline trajectory optimization](https://mlsanity.com/paper/2209.07003/) (2022)
- [Autonomous Flights in Dynamic Environments with Onboard Vision](https://mlsanity.com/paper/2103.05870/) (2021)
- [Flying on Point Clouds with Reinforcement Learning](https://mlsanity.com/paper/2503.00496/) (2025)
- [Risk-aware Trajectory Sampling for Quadrotor Obstacle Avoidance in Dynamic Environments](https://mlsanity.com/paper/2201.06645/) (2022)
- [FAPP: Fast and Adaptive Perception and Planning for UAVs in Dynamic Cluttered Environments](https://mlsanity.com/paper/2312.08743/) (2023)
- [Flying in Highly Dynamic Environments with End-to-end Learning Approach](https://mlsanity.com/paper/2503.14352/) (2025)
- [Spatiotemporal Attention Enhances Lidar-Based Robot Navigation in Dynamic Environments](https://mlsanity.com/paper/2310.19670/) (2023)
- [Flow-Aided Flight Through Dynamic Clutters From Point To Motion](https://mlsanity.com/paper/2511.16372/) (2025)
- [Intent Prediction-Driven Model Predictive Control for UAV Planning and Navigation in Dynamic Environments](https://mlsanity.com/paper/2409.15633/) (2024)

## Abstract (arXiv)

Safe quadrotor navigation in cluttered and dynamic environments depends not only on instantaneous geometric perception, but more critically on anticipating collision risks induced by relative motion. Conventional modular pipelines frequently suffer from perception latency, while end-to-end learning methods relying on implicit scalar rewards often struggle to extract reliable spatio-temporal features without physics-grounded supervision. To address this, we propose an anticipatory risk-guided reinforcement learning framework. Leveraging privileged simulator states, we construct a directionally aligned future collision risk map based on the Closest Point of Approach (CPA). Through an asymmetric actor-critic architecture, the network is trained to self-predict this structured risk, which explicitly guides the visual policy during deployment. A lightweight spatio-temporal encoder extracts motion cues directly from onboard depth sequences, bypassing explicit object tracking or optical flow estimation. Extensive simulated and real-world experiments demonstrate that our method effectively improves safety margins and flight efficiency in dense dynamic clutters compared to existing baselines. Furthermore, the learned policy achieves robust zero-shot Sim-to-Real transfer on a physical quadrotor, relying purely on abstracted spatio-temporal depth sequences and its self-predicted risk priors, validating the effectiveness of our approach and its robust generalization from simulation to reality.