# Energy-aware optimization of UAV base stations placement via decentralized multi-agent Q-learning

arXiv: https://arxiv.org/abs/2106.00845 · HTML: https://mlsanity.com/paper/2106.00845/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 8 of 8)

- [Multi-Agent Reinforcement Learning Based Resource Allocation for UAV Networks](https://mlsanity.com/paper/1810.10408/) (2018)
- [Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks](https://mlsanity.com/paper/1705.02723/) (2017)
- [Reinforcement Learning in Multiple-UAV Networks: Deployment and Movement Design](https://mlsanity.com/paper/1904.05242/) (2019)
- [Efficient 3-D Placement of an Aerial Base Station in Next Generation Cellular Networks](https://mlsanity.com/paper/1603.00300/) (2016)
- [Energy Efficient UAV Communication with Energy Harvesting](https://mlsanity.com/paper/1911.00351/) (2019)
- [REQIBA: Regression and Deep Q-Learning for Intelligent UAV Cellular User to Base Station Association](https://mlsanity.com/paper/2010.01126/) (2020)
- [A Stochastic Model for UAV Networks Positioned Above Demand Hotspots in Urban Environments](https://mlsanity.com/paper/1804.11001/) (2018)
- [Coverage Analysis for Low-Altitude UAV Networks in Urban Environments](https://mlsanity.com/paper/1704.06214/) (2017)

## Built on by (most cited first, 3 of 3)

- [Optimising Energy Efficiency in UAV-Assisted Networks using Deep Reinforcement Learning](https://mlsanity.com/paper/2204.01597/) (2022)
- [Density-Aware Reinforcement Learning to Optimise Energy Efficiency in UAV-Assisted Networks](https://mlsanity.com/paper/2306.08785/) (2023)
- [Pareto Q-Learning with Reward Machines](https://mlsanity.com/paper/2606.19134/) (2026)

## Abstract (arXiv)

Unmanned aerial vehicles serving as aerial base stations (UAV-BSs) can be deployed to provide wireless connectivity to ground devices in events of increased network demand, points-of-failure in existing infrastructure, or disasters. However, it is challenging to conserve the energy of UAVs during prolonged coverage tasks, considering their limited on-board battery capacity. Reinforcement learning-based (RL) approaches have been previously used to improve energy utilization of multiple UAVs, however, a central cloud controller is assumed to have complete knowledge of the end-devices' locations, i.e., the controller periodically scans and sends updates for UAV decision-making. This assumption is impractical in dynamic network environments with UAVs serving mobile ground devices. To address this problem, we propose a decentralized Q-learning approach, where each UAV-BS is equipped with an autonomous agent that maximizes the connectivity of mobile ground devices while improving its energy utilization. Experimental results show that the proposed design significantly outperforms the centralized approaches in jointly maximizing the number of connected ground devices and the energy utilization of the UAV-BSs.