# Dynamic Q-planning for Online UAV Path Planning in Unknown and Complex Environments

arXiv: https://arxiv.org/abs/2402.06297 · HTML: https://mlsanity.com/paper/2402.06297/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 68 references that are in the map.
- First posted on arXiv 9 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 14 of 14)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Uncertainty-Aware Reinforcement Learning for Collision Avoidance](https://mlsanity.com/paper/1702.01182/) (2017)
- [Autonomous Drone Racing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.08624/) (2021)
- [Memory-based Deep Reinforcement Learning for Obstacle Avoidance in UAV with Limited Environment Knowledge](https://mlsanity.com/paper/1811.03307/) (2018)
- [Time-Optimal Planning for Quadrotor Waypoint Flight](https://mlsanity.com/paper/2108.04537/) (2021)
- [3D Simulation for Robot Arm Control with Deep Q-Learning](https://mlsanity.com/paper/1609.03759/) (2016)
- [The MRS UAV System: Pushing the Frontiers of Reproducible Research, Real-world Deployment, and Education with Autonomous Unmanned Aerial Vehicles](https://mlsanity.com/paper/2008.08050/) (2020)
- [PathBench: A Benchmarking Platform for Classical and Learned Path Planning Algorithms](https://mlsanity.com/paper/2105.01777/) (2021)
- [Online Exploration and Coverage Planning in Unknown Obstacle-Cluttered Environments](https://mlsanity.com/paper/2006.16460/) (2020)
- [Fast Simulation of Crowd Collision Avoidance](https://mlsanity.com/paper/1908.10107/) (2019)
- [UAV Aided Search and Rescue Operation Using Reinforcement Learning](https://mlsanity.com/paper/2002.08415/) (2020)

## Abstract (arXiv)

Unmanned Aerial Vehicles need an online path planning capability to move in high-risk missions in unknown and complex environments to complete them safely. However, many algorithms reported in the literature may not return reliable trajectories to solve online problems in these scenarios. The Q-Learning algorithm, a Reinforcement Learning Technique, can generate trajectories in real-time and has demonstrated fast and reliable results. This technique, however, has the disadvantage of defining the iteration number. If this value is not well defined, it will take a long time or not return an optimal trajectory. Therefore, we propose a method to dynamically choose the number of iterations to obtain the best performance of Q-Learning. The proposed method is compared to the Q-Learning algorithm with a fixed number of iterations, A*, Rapid-Exploring Random Tree, and Particle Swarm Optimization. As a result, the proposed Q-learning algorithm demonstrates the efficacy and reliability of online path planning with a dynamic number of iterations to carry out online missions in unknown and complex environments.