# Hierarchical Multi-Agent DRL-Based Framework for Joint Multi-RAT Assignment and Dynamic Resource Allocation in Next-Generation HetNets

arXiv: https://arxiv.org/abs/2202.13652 · HTML: https://mlsanity.com/paper/2202.13652/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 28 Feb 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Applications of Deep Reinforcement Learning in Communications and Networking: A Survey](https://mlsanity.com/paper/1810.07862/) (2018)
- [A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems](https://mlsanity.com/paper/1902.10265/) (2019)
- [Machine Learning for Resource Management in Cellular and IoT Networks: Potentials, Current Solutions, and Open Challenges](https://mlsanity.com/paper/1907.08965/) (2019)
- [Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination](https://mlsanity.com/paper/1907.00123/) (2019)
- [DeepSlicing: Deep Reinforcement Learning Assisted Resource Allocation for Network Slicing](https://mlsanity.com/paper/2008.07614/) (2020)
- [Deep Reinforcement Learning for Joint Spectrum and Power Allocation in Cellular Networks](https://mlsanity.com/paper/2012.10682/) (2020)
- [Intelligent User Association for Symbiotic Radio Networks using Deep Reinforcement Learning](https://mlsanity.com/paper/1905.04041/) (2019)
- [DQN-Based Multi-User Power Allocation for Hybrid RF/VLC Networks](https://mlsanity.com/paper/2102.01884/) (2021)

## Built on by (most cited first, 1 of 1)

- [Hierarchical Multi-Agent DRL Based Dynamic Cluster Reconfiguration for UAV Mobility Management](https://mlsanity.com/paper/2412.16167/) (2024)

## Abstract (arXiv)

This paper considers the problem of cost-aware downlink sum-rate maximization via joint optimal radio access technologies (RATs) assignment and power allocation in next-generation heterogeneous wireless networks (HetNets). We consider a future HetNet comprised of multi-RATs and serving multi-connectivity edge devices (EDs), and we formulate the problem as mixed-integer non-linear programming (MINP) problem. Due to the high complexity and combinatorial nature of this problem and the difficulty to solve it using conventional methods, we propose a hierarchical multi-agent deep reinforcement learning (DRL)-based framework, called DeepRAT, to solve it efficiently and learn system dynamics. In particular, the DeepRAT framework decomposes the problem into two main stages; the RATs-EDs assignment stage, which implements a single-agent Deep Q Network (DQN) algorithm, and the power allocation stage, which utilizes a multi-agent Deep Deterministic Policy Gradient (DDPG) algorithm. Using simulations, we demonstrate how the various DRL agents efficiently interact to learn system dynamics and derive the global optimal policy. Furthermore, our simulation results show that the proposed DeepRAT algorithm outperforms existing state-of-the-art heuristic approaches in terms of network utility. Finally, we quantitatively show the ability of the DeepRAT model to quickly and dynamically adapt to abrupt changes in network dynamics, such as EDs mobility.