# Navigation in a Three-Dimensional Urban Flow using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2510.25679 · HTML: https://mlsanity.com/paper/2510.25679/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 45 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Stabilizing Transformers for Reinforcement Learning](https://mlsanity.com/paper/1910.06764/) (2019)
- [A Review on Deep Learning in UAV Remote Sensing](https://mlsanity.com/paper/2101.10861/) (2021)
- [Learning Efficient Navigation in Vortical Flow Fields](https://mlsanity.com/paper/2102.10536/) (2021)
- [A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidance](https://mlsanity.com/paper/2103.06403/) (2021)
- [Autonomous UAV Navigation: A DDPG-based Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2003.10923/) (2020)
- [Navigation in a simplified Urban Flow through Deep Reinforcement Learning](https://mlsanity.com/paper/2409.17922/) (2024)
- [Learning-Based UAV Trajectory Optimization with Collision Avoidance and Connectivity Constraints](https://mlsanity.com/paper/2104.06256/) (2021)
- [Path Planning using Instruction-Guided Probabilistic Roadmaps](https://mlsanity.com/paper/2502.16515/) (2025)
- [FR-SLAM: A SLAM Improvement Method Based on Floor Plan Registration](https://mlsanity.com/paper/2407.11299/) (2024)

## Abstract (arXiv)

Unmanned Aerial Vehicles (UAVs) are increasingly populating urban areas for delivery and surveillance purposes. In this work, we develop an optimal navigation strategy based on Deep Reinforcement Learning. The environment is represented by a three-dimensional high-fidelity simulation of an urban flow, characterized by turbulence and recirculation zones. The algorithm presented here is a flow-aware Proximal Policy Optimization (PPO) combined with a Gated Transformer eXtra Large (GTrXL) architecture, giving the agent richer information about the turbulent flow field in which it navigates. The results are compared with a PPO+GTrXL without the secondary prediction tasks, a PPO combined with Long Short Term Memory (LSTM) cells and a traditional navigation algorithm. The obtained results show a significant increase in the success rate (SR) and a lower crash rate (CR) compared to a PPO+LSTM, PPO+GTrXL and the classical Zermelo's navigation algorithm, paving the way to a completely reimagined UAV landscape in complex urban environments.