# Multi-Agent Q-Learning for Real-Time Load Balancing User Association and Handover in Mobile Networks

arXiv: https://arxiv.org/abs/2412.19835 · HTML: https://mlsanity.com/paper/2412.19835/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 22 Dec 2024.

## Builds on (most cited first, 10 of 10)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks](https://mlsanity.com/paper/1812.09755/) (2018)
- [Multi-Agent Reinforcement Learning for Adaptive User Association in Dynamic mmWave Networks](https://mlsanity.com/paper/2006.09066/) (2020)
- [Intelligent User Association for Symbiotic Radio Networks using Deep Reinforcement Learning](https://mlsanity.com/paper/1905.04041/) (2019)
- [Optimal User-Cell Association for Massive MIMO Wireless Networks](https://mlsanity.com/paper/1407.6731/) (2014)
- [Learning-based Load Balancing Handover in Mobile Millimeter Wave Networks](https://mlsanity.com/paper/2011.01420/) (2020)
- [Distributed User Association in B5G Networks Using Early Acceptance Matching Games](https://mlsanity.com/paper/2012.01510/) (2020)

## Abstract (arXiv)

As next generation cellular networks become denser, associating users with the optimal base stations at each time while ensuring no base station is overloaded becomes critical for achieving stable and high network performance. We propose multi-agent online Q-learning (QL) algorithms for performing real-time load balancing user association and handover in dense cellular networks. The load balancing constraints at all base stations couple the actions of user agents, and we propose two multi-agent action selection policies, one centralized and one distributed, to satisfy load balancing at every learning step. In the centralized policy, the actions of UEs are determined by a central load balancer (CLB) running an algorithm based on swapping the worst connection to maximize the total learning reward. In the distributed policy, each UE takes an action based on its local information by participating in a distributed matching game with the BSs to maximize the local reward. We then integrate these action selection policies into an online QL algorithm that adapts in real-time to network dynamics including channel variations and user mobility, using a reward function that considers a handover cost to reduce handover frequency. The proposed multi-agent QL algorithm features low-complexity and fast convergence, outperforming 3GPP max-SINR association. Both policies adapt well to network dynamics at various UE speed profiles from walking, running, to biking and suburban driving, illustrating their robustness and real-time adaptability.