mlsanityData to 7 Oct 2026

Navigation in a simplified Urban Flow through Deep Reinforcement Learning

arXiv 2409.17922 ↗First posted 26 Sep 2024cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

The increasing number of unmanned aerial vehicles (UAVs) in urban environments requires a strategy to minimize their environmental impact, both in terms of energy efficiency and noise reduction. In order to reduce these concerns, novel strategies for developing prediction models and optimization of flight planning, for instance through deep reinforcement learning (DRL), are needed. Our goal is to develop DRL algorithms capable of enabling the autonomous navigation of UAVs in urban environments, taking into account the presence of buildings and other UAVs, optimizing the trajectories in order to reduce both energetic consumption and noise. This is achieved using fluid-flow simulations which represent the environment in which UAVs navigate and training the UAV as an agent interacting with an urban environment. In this work, we consider a domain domain represented by a two-dimensional flow field with obstacles, ideally representing buildings, extracted from a three-dimensional high-fidelity numerical simulation. The presented methodology, using PPO+LSTM cells, was validated by reproducing a simple but fundamental problem in navigation, namely the Zermelo's problem, which deals with a vessel navigating in a turbulent flow, travelling from a starting point to a target location, optimizing the trajectory. The current method shows a significant improvement with respect to both a simple PPO and a TD3 algorithm, with a success rate (SR) of the PPO+LSTM trained policy of 98.7%, and a crash rate (CR) of 0.1%, outperforming both PPO (SR = 75.6%, CR=18.6%) and TD3 (SR=77.4% and CR=14.5%). This is the first step towards DRL strategies which will guide UAVs in a three-dimensional flow field using real-time signals, making the navigation efficient in terms of flight time and avoiding damages to the vehicle.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 20 of its 69 references that are in the map. First posted on arXiv 26 Sep 2024.

Builds on

20 of 69 references.

References · 20

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  5. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  6. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  7. 2021Enhancing Computational Fluid Dynamics with Machine Learningcited by 137
  8. 2018PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Makingcited by 51
  9. 2021A Review on Deep Learning in UAV Remote Sensingcited by 32
  10. 2021Learning Efficient Navigation in Vortical Flow Fieldscited by 26
  11. 2023Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you needcited by 25
  12. 2018Memory-based Deep Reinforcement Learning for Obstacle Avoidance in UAV with Limited Environment Knowledgecited by 21
8 more
  1. 2019Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learningcited by 21
  2. 2021A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidancecited by 8
  3. 2022Emerging trends in machine learning for computational fluid dynamicscited by 7
  4. 2023Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensingcited by 7
  5. 2019Effects of a Hovering Unmanned Aerial Vehicle on Urban Soundscapes Perceptioncited by 3
  6. 2023Active flow control for three-dimensional cylinders through deep reinforcement learningcited by 3
  7. 2021Importance of Environment Design in Reinforcement Learning: A Study of a Robotic Environmentcited by 1
  8. 2023An Integrated Real-time UAV Trajectory Optimization with Potential Field Approach for Dynamic Collision Avoidancecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2024Transformer-Based Fault-Tolerant Control for Fixed-Wing UAVs Using Knowledge Distillation and In-Context Adaptationcited by 3
  2. 2024Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order modelscited by 0
  3. 2025Navigation in a Three-Dimensional Urban Flow using Deep Reinforcement Learningcited by 0
  4. 2026Timescale Separation Enables Deep Reinforcement Learning Control of Rotating Detonation Engine Mode Transitionscited by 0
  5. 2026Explainable deep reinforcement learning reveals energy-efficient control strategies for turbulent drag reductioncited by 0