# Multi-Agent Reinforcement Learning with Action Masking for UAV-enabled Mobile Communications

arXiv: https://arxiv.org/abs/2303.16737 · HTML: https://mlsanity.com/paper/2303.16737/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 62 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 15 of 15)

- [A Closer Look at Invalid Action Masking in Policy Gradient Algorithms](https://mlsanity.com/paper/2006.14171/) (2020)
- [Mastering Complex Control in MOBA Games with Deep Reinforcement Learning](https://mlsanity.com/paper/1912.09729/) (2019)
- [Learning to Perform Physics Experiments via Deep Reinforcement Learning](https://mlsanity.com/paper/1611.01843/) (2016)
- [Wireless Communications with Unmanned Aerial Vehicles: Opportunities and Challenges](https://mlsanity.com/paper/1602.03602/) (2016)
- [Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues](https://mlsanity.com/paper/1809.08707/) (2018)
- [A Survey on Model-based, Heuristic, and Machine Learning Optimization Approaches in RIS-aided Wireless Networks](https://mlsanity.com/paper/2303.14320/) (2023)
- [Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV based Random Access IoT Networks with NOMA](https://mlsanity.com/paper/2002.00073/) (2020)
- [Optimal 3D-Trajectory Design and Resource Allocation for Solar-Powered UAV Communication Systems](https://mlsanity.com/paper/1808.00101/) (2018)
- [Cellular UAV-to-X Communications: Design and Optimization for Multi-UAV Networks](https://mlsanity.com/paper/1801.05000/) (2018)
- [Robust Secure UAV Communications with the Aid of Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/2008.09404/) (2020)
- [Multi-Agent Reinforcement Learning in NOMA-aided UAV Networks for Cellular Offloading](https://mlsanity.com/paper/2010.09094/) (2020)
- [Non-Orthogonal Multiple Access (NOMA) in Cellular Uplink and Downlink: Challenges and Enabling Techniques](https://mlsanity.com/paper/1608.05783/) (2016)
- [Hybrid Offline-Online Design for UAV-Enabled Data Harvesting in Probabilistic LoS Channel](https://mlsanity.com/paper/1907.06181/) (2019)
- [Energy-Efficient NOMA Enabled Heterogeneous Cloud Radio Access Networks](https://mlsanity.com/paper/1801.01996/) (2018)
- [Joint Power Control and User Association for NOMA-Based Full-Duplex Systems](https://mlsanity.com/paper/1908.00833/) (2019)

## Built on by (most cited first, 2 of 2)

- [MA-HEAD-Net: Adaptive Rule-Guided Multi-Agent DRL for AoI Minimization in UAV-Assisted Emergency Networks](https://mlsanity.com/paper/2608.01128/) (2026)
- [Hybrid Belief Reinforcement Learning for Efficient Coordinated Spatial Exploration](https://mlsanity.com/paper/2603.03595/) (2026)

## Abstract (arXiv)

Unmanned Aerial Vehicles (UAVs) are increasingly used as aerial base stations to provide ad hoc communications infrastructure. Building upon prior research efforts which consider either static nodes, 2D trajectories or single UAV systems, this paper focuses on the use of multiple UAVs for providing wireless communication to mobile users in the absence of terrestrial communications infrastructure. In particular, we jointly optimize UAV 3D trajectory and NOMA power allocation to maximize system throughput. Firstly, a weighted K-means-based clustering algorithm establishes UAV-user associations at regular intervals. The efficacy of training a novel Shared Deep Q-Network (SDQN) with action masking is then explored. Unlike training each UAV separately using DQN, the SDQN reduces training time by using the experiences of multiple UAVs instead of a single agent. We also show that SDQN can be used to train a multi-agent system with differing action spaces. Simulation results confirm that: 1) training a shared DQN outperforms a conventional DQN in terms of maximum system throughput (+20%) and training time (-10%); 2) it can converge for agents with different action spaces, yielding a 9% increase in throughput compared to mutual learning algorithms; and 3) combining NOMA with an SDQN architecture enables the network to achieve a better sum rate compared with existing baseline schemes.