# Hierarchical Multi-Agent DRL Based Dynamic Cluster Reconfiguration for UAV Mobility Management

arXiv: https://arxiv.org/abs/2412.16167 · HTML: https://mlsanity.com/paper/2412.16167/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 42 references that are in the map.
- First posted on arXiv 5 Dec 2024.

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications](https://mlsanity.com/paper/1812.11794/) (2018)
- [Distributed Multi-agent Meta Learning for Trajectory Design in Wireless Drone Networks](https://mlsanity.com/paper/2012.03158/) (2020)
- [Making Cell-Free Massive MIMO Competitive With MMSE Processing and Centralized Implementation](https://mlsanity.com/paper/1903.10611/) (2019)
- [Efficient Drone Mobility Support Using Reinforcement Learning](https://mlsanity.com/paper/1911.09715/) (2019)
- [Joint Pilot and Payload Power Allocation for Massive-MIMO-enabled URLLC IIoT Networks](https://mlsanity.com/paper/1912.12438/) (2019)
- [Cell-Free Massive MIMO in O-RAN: Energy-Aware Joint Orchestration of Cloud, Fronthaul, and Radio Resources](https://mlsanity.com/paper/2301.06166/) (2023)
- [Machine Learning assisted Handover and Resource Management for Cellular Connected Drones](https://mlsanity.com/paper/2001.07937/) (2020)
- [REQIBA: Regression and Deep Q-Learning for Intelligent UAV Cellular User to Base Station Association](https://mlsanity.com/paper/2010.01126/) (2020)
- [EnTRPO: Trust Region Policy Optimization Method with Entropy Regularization](https://mlsanity.com/paper/2110.13373/) (2021)
- [Cell-Free Massive MIMO in the O-RAN Architecture: Cluster and Handover Strategies](https://mlsanity.com/paper/2301.07618/) (2023)
- [3D beamforming and handover analysis for UAV networks](https://mlsanity.com/paper/2010.03413/) (2020)
- [Hierarchical Multi-Agent DRL-Based Framework for Joint Multi-RAT Assignment and Dynamic Resource Allocation in Next-Generation HetNets](https://mlsanity.com/paper/2202.13652/) (2022)
- [Cell-Free Massive MIMO for URLLC: A Finite-Blocklength Analysis](https://mlsanity.com/paper/2207.00856/) (2022)
- [Reinforcement Learning Based Dynamic Power Control for UAV Mobility Management](https://mlsanity.com/paper/2312.04742/) (2023)
- [Learning Based Dynamic Cluster Reconfiguration for UAV Mobility Management with 3D Beamforming](https://mlsanity.com/paper/2402.00224/) (2024)

## Built on by (most cited first, 3 of 3)

- [Spatiotemporal Continual Federated Learning for Agentic Multi-UAV Edge Networks: Mitigating Catastrophic Forgetting](https://mlsanity.com/paper/2601.21861/) (2026)
- [Demystifying Deep Reinforcement Learning: A Neuro-Symbolic Framework for Interpretable Open RAN Automation](https://mlsanity.com/paper/2605.10648/) (2026)
- [DSWM: Decomposed Spatio-Temporal World Model for Demand-Driven UAV Base Station Repositioning](https://mlsanity.com/paper/2609.36845/) (2026)

## Abstract (arXiv)

Multi-connectivity involves dynamic cluster formation among distributed access points (APs) and coordinated resource allocation from these APs, highlighting the need for efficient mobility management strategies for users with multi-connectivity. In this paper, we propose a novel mobility management scheme for unmanned aerial vehicles (UAVs) that uses dynamic cluster reconfiguration with energy-efficient power allocation in a wireless interference network. Our objective encompasses meeting stringent reliability demands, minimizing joint power consumption, and reducing the frequency of cluster reconfiguration. To achieve these objectives, we propose a hierarchical multi-agent deep reinforcement learning (H-MADRL) framework, specifically tailored for dynamic clustering and power allocation. The edge cloud connected with a set of APs through low latency optical back-haul links hosts the high-level agent responsible for the optimal clustering policy, while low-level agents reside in the APs and are responsible for the power allocation policy. To further improve the learning efficiency, we propose a novel action-observation transition-driven learning algorithm that allows the low-level agents to use the action space from the high-level agent as part of the local observation space. This allows the lower-level agents to share partial information about the clustering policy and allocate the power more efficiently. The simulation results demonstrate that our proposed distributed algorithm achieves comparable performance to the centralized algorithm. Additionally, it offers better scalability, as the decision time for clustering and power allocation increases by only 10% when doubling the number of APs, compared to a 90% increase observed with the centralized approach.