# Multi-Agent Reinforcement Learning Based Resource Allocation for UAV Networks

arXiv: https://arxiv.org/abs/1810.10408 · HTML: https://mlsanity.com/paper/1810.10408/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2018.

## Built on by (most cited first, 30 of 56)

- [Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey](https://mlsanity.com/paper/2110.13484/) (2021)
- [Towards Comprehensive Testing on the Robustness of Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2204.07932/) (2022)
- [UAV-assisted Online Machine Learning over Multi-Tiered Networks: A Hierarchical Nested Personalized Federated Learning Approach](https://mlsanity.com/paper/2106.15734/) (2021)
- [Group-Aware Coordination Graph for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2404.10976/) (2024)
- [Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Survey](https://mlsanity.com/paper/2504.21048/) (2025)
- [UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVs](https://mlsanity.com/paper/2506.08045/) (2025)
- [Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2402.08421/) (2024)
- [Toward Enhanced Reinforcement Learning-Based Resource Management via Digital Twin: Opportunities, Applications, and Challenges](https://mlsanity.com/paper/2406.07857/) (2024)
- [Aerial Reliable Collaborative Communications for Terrestrial Mobile Users via Evolutionary Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/2502.05824/) (2025)
- [Achieving Fairness in Multi-Agent Markov Decision Processes Using Reinforcement Learning](https://mlsanity.com/paper/2306.00324/) (2023)
- [Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2403.19253/) (2024)
- [Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specifications](https://mlsanity.com/paper/2306.06808/) (2023)
- [Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for mmWave Multiuser MIMO with Lens Arrays](https://mlsanity.com/paper/2101.01336/) (2021)
- [Distributed Machine Learning for UAV Swarms: Computing, Sensing, and Semantics](https://mlsanity.com/paper/2301.00912/) (2023)
- [CE-NAS: An End-to-End Carbon-Efficient Neural Architecture Search Framework](https://mlsanity.com/paper/2406.01414/) (2024)
- [Semantic-Aware Resource Management for C-V2X Platooning via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2411.04672/) (2024)
- [Resource allocation in dynamic multiagent systems](https://mlsanity.com/paper/2102.08317/) (2021)
- [Deep Reinforcement Learning based Autonomous Decision-Making for Cooperative UAVs: A Search and Rescue Real World Application](https://mlsanity.com/paper/2502.20326/) (2025)
- [Optimizing Communication and Device Clustering for Clustered Federated Learning with Differential Privacy](https://mlsanity.com/paper/2507.07320/) (2025)
- [Learning based signal detection for MIMO systems with unknown noise statistics](https://mlsanity.com/paper/2101.08435/) (2021)
- [Distributed Reinforcement Learning for Flexible and Efficient UAV Swarm Control](https://mlsanity.com/paper/2103.04666/) (2021)
- [Energy-aware optimization of UAV base stations placement via decentralized multi-agent Q-learning](https://mlsanity.com/paper/2106.00845/) (2021)
- [Cooperative Multi-Agent Reinforcement Learning with Hypergraph Convolution](https://mlsanity.com/paper/2112.06771/) (2021)
- [Decentralized Federated Reinforcement Learning for User-Centric Dynamic TFDD Control](https://mlsanity.com/paper/2211.02296/) (2022)
- [UAV-Assisted Space-Air-Ground Integrated Networks: A Technical Review of Recent Learning Algorithms](https://mlsanity.com/paper/2211.14931/) (2022)
- [Reliable and Efficient Data Collection in UAV-based IoT Networks](https://mlsanity.com/paper/2311.05303/) (2023)
- [Latency Minimization for UAV-Enabled Federated Learning: Trajectory Design and Resource Allocation](https://mlsanity.com/paper/2412.07428/) (2024)
- [Machine Learning Methods for Management UAV Flocks -- a Survey](https://mlsanity.com/paper/2108.13448/) (2021)
- [Distributed Control using Reinforcement Learning with Temporal-Logic-Based Reward Shaping](https://mlsanity.com/paper/2203.04172/) (2022)
- [Learning-Based Client Selection for Federated Learning Services Over Wireless Networks with Constrained Monetary Budgets](https://mlsanity.com/paper/2208.04322/) (2022)

## Abstract (arXiv)

Unmanned aerial vehicles (UAVs) are capable of serving as aerial base stations (BSs) for providing both cost-effective and on-demand wireless communications. This article investigates dynamic resource allocation of multiple UAVs enabled communication networks with the goal of maximizing long-term rewards. More particularly, each UAV communicates with a ground user by automatically selecting its communicating users, power levels and subchannels without any information exchange among UAVs. To model the uncertainty of environments, we formulate the long-term resource allocation problem as a stochastic game for maximizing the expected rewards, where each UAV becomes a learning agent and each resource allocation solution corresponds to an action taken by the UAVs. Afterwards, we develop a multi-agent reinforcement learning (MARL) framework that each agent discovers its best strategy according to its local observations using learning. More specifically, we propose an agent-independent method, for which all agents conduct a decision algorithm independently but share a common structure based on Q-learning. Finally, simulation results reveal that: 1) appropriate parameters for exploitation and exploration are capable of enhancing the performance of the proposed MARL based resource allocation algorithm; 2) the proposed MARL algorithm provides acceptable performance compared to the case with complete information exchanges among UAVs. By doing so, it strikes a good tradeoff between performance gains and information exchange overheads.