# Hierarchical Deep Q-Learning Based Handover in Wireless Networks with Dual Connectivity

arXiv: https://arxiv.org/abs/2301.05391 · HTML: https://mlsanity.com/paper/2301.05391/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 13 Jan 2023.

## Builds on (most cited first, 6 of 6)

- [Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation](https://mlsanity.com/paper/1604.06057/) (2016)
- [AI-enabled Future Wireless Networks: Challenges, Opportunities and Open Issues](https://mlsanity.com/paper/2103.04536/) (2021)
- [End-to-End Simulation of 5G mmWave Networks](https://mlsanity.com/paper/1705.02882/) (2017)
- [Initial Access in 5G mm-Wave Cellular Networks](https://mlsanity.com/paper/1602.07731/) (2016)
- [Improved Handover Through Dual Connectivity in 5G mmWave Mobile Networks](https://mlsanity.com/paper/1611.04748/) (2016)
- [QoS-Aware Load Balancing in Wireless Networks using Clipped Double Q-Learning](https://mlsanity.com/paper/2107.04207/) (2021)

## Built on by (most cited first, 1 of 1)

- [Deep Reinforcement Learning Approach to QoSAware Load Balancing in 5G Cellular Networks under User Mobility and Observation Uncertainty](https://mlsanity.com/paper/2510.24869/) (2025)

## Abstract (arXiv)

5G New Radio proposes the usage of frequencies above 10 GHz to speed up LTE's existent maximum data rates. However, the effective size of 5G antennas and consequently its repercussions in the signal degradation in urban scenarios makes it a challenge to maintain stable coverage and connectivity. In order to obtain the best from both technologies, recent dual connectivity solutions have proved their capabilities to improve performance when compared with coexistent standalone 5G and 4G technologies. Reinforcement learning (RL) has shown its huge potential in wireless scenarios where parameter learning is required given the dynamic nature of such context. In this paper, we propose two reinforcement learning algorithms: a single agent RL algorithm named Clipped Double Q-Learning (CDQL) and a hierarchical Deep Q-Learning (HiDQL) to improve Multiple Radio Access Technology (multi-RAT) dual-connectivity handover. We compare our proposal with two baselines: a fixed parameter and a dynamic parameter solution. Simulation results reveal significant improvements in terms of latency with a gain of 47.6% and 26.1% for Digital-Analog beamforming (BF), 17.1% and 21.6% for Hybrid-Analog BF, and 24.7% and 39% for Analog-Analog BF when comparing the RL-schemes HiDQL and CDQL with the with the existent solutions, HiDQL presented a slower convergence time, however obtained a more optimal solution than CDQL. Additionally, we foresee the advantages of utilizing context-information as geo-location of the UEs to reduce the beam exploration sector, and thus improving further multi-RAT handover latency results.