# Beam Management in Ultra-dense mmWave Network via Federated Reinforcement Learning: An Intelligent and Secure Approach

arXiv: https://arxiv.org/abs/2210.01307 · HTML: https://mlsanity.com/paper/2210.01307/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 39 references that are in the map.
- First posted on arXiv 4 Oct 2022.

## Builds on (most cited first, 8 of 8)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [Federated Learning for Wireless Communications: Motivation, Opportunities and Challenges](https://mlsanity.com/paper/1908.06847/) (2019)
- [A Tutorial on Beam Management for 3GPP NR at mmWave Frequencies](https://mlsanity.com/paper/1804.01908/) (2018)
- [Millimeter Wave Channel Modeling and Cellular Capacity Evaluation](https://mlsanity.com/paper/1312.4921/) (2013)
- [Standalone and Non-Standalone Beam Management for 3GPP NR at mmWaves](https://mlsanity.com/paper/1805.04268/) (2018)
- [Beamspace SU-MIMO for Future Millimeter Wave Wireless Communications](https://mlsanity.com/paper/1705.05215/) (2017)
- [Beam Management for Millimeter Wave Beamspace MU-MIMO Systems](https://mlsanity.com/paper/1710.03640/) (2017)

## Built on by (most cited first, 3 of 3)

- [Pay Less But Get More: A Dual-Attention-based Channel Estimation Network for Massive MIMO Systems with Low-Density Pilots](https://mlsanity.com/paper/2303.00986/) (2023)
- [Digital Twin-Assisted Explainable AI for Robust Beam Prediction in mmWave MIMO Systems](https://mlsanity.com/paper/2507.14180/) (2025)
- [Hierarchical ML Codebook Design for Extreme MIMO Beam Management](https://mlsanity.com/paper/2312.02178/) (2023)

## Abstract (arXiv)

Deploying ultra-dense networks that operate on millimeter wave (mmWave) band is a promising way to address the tremendous growth on mobile data traffic. However, one key challenge of ultra-dense mmWave network (UDmmN) is beam management due to the high propagation delay, limited beam coverage as well as numerous beams and users. In this paper, a novel systematic beam control scheme is presented to tackle the beam management problem which is difficult due to the nonconvex objective function. We employ double deep Q-network (DDQN) under a federated learning (FL) framework to address the above optimization problem, and thereby fulfilling adaptive and intelligent beam management in UDmmN. In the proposed beam management scheme based on FL (BMFL), the non-rawdata aggregation can theoretically protect user privacy while reducing handoff cost. Moreover, we propose to adopt a data cleaning technique in the local model training for BMFL, with the aim to further strengthen the privacy protection of users while improving the learning convergence speed. Simulation results demonstrate the performance gain of our proposed scheme.