# Meta-Learning Multi-armed Bandits for Beam Tracking in 5G and 6G Networks

arXiv: https://arxiv.org/abs/2512.05680 · HTML: https://mlsanity.com/paper/2512.05680/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 5 Dec 2025.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Gradient Estimation Using Stochastic Computation Graphs](https://mlsanity.com/paper/1506.05254/) (2015)
- [Regret Bounds for Thompson Sampling in Episodic Restless Bandit Problems](https://mlsanity.com/paper/1905.12673/) (2019)
- [A Tutorial on Beam Management for 3GPP NR at mmWave Frequencies](https://mlsanity.com/paper/1804.01908/) (2018)
- [Parallelizing Thompson Sampling](https://mlsanity.com/paper/2106.01420/) (2021)
- [Thompson Sampling in Non-Episodic Restless Bandits](https://mlsanity.com/paper/1910.05654/) (2019)
- [Optimal Recommendation to Users that React: Online Learning for a Class of POMDPs](https://mlsanity.com/paper/1603.09233/) (2016)
- [Shrinkage Estimators in Online Experiments](https://mlsanity.com/paper/1904.12918/) (2019)
- [Deep Reinforcement Learning based Blind mmWave MIMO Beam Alignment](https://mlsanity.com/paper/2001.09251/) (2020)
- [A Hidden Markov Restless Multi-armed Bandit Model for Playout Recommendation Systems](https://mlsanity.com/paper/1704.02894/) (2017)
- [Deep reinforcement learning approach to MIMO precoding problem: Optimality and Robustness](https://mlsanity.com/paper/2006.16646/) (2020)
- [Restless Multi-Armed Bandits under Exogenous Global Markov Process](https://mlsanity.com/paper/2202.13665/) (2022)
- [Attenuation of Several Common Building Materials in Millimeter-Wave Frequency Bands: 28, 73 and 91 GHz](https://mlsanity.com/paper/2004.12568/) (2020)
- [To RL or not to RL? An Algorithmic Cheat-Sheet for AI-Based Radio Resource Management](https://mlsanity.com/paper/2405.19045/) (2024)

## Abstract (arXiv)

Beamforming-capable antenna arrays with many elements enable higher data rates in next generation 5G and 6G networks. In current practice, analog beamforming uses a codebook of pre-configured beams with each of them radiating towards a specific direction, and a beam management function continuously selects textitoptimal beams for moving user equipments (UEs). However, large codebooks and effects caused by reflections or blockages of beams make an optimal beam selection challenging. In contrast to previous work and standardization efforts that opt for supervised learning to train classifiers to predict the next best beam based on previously selected beams we formulate the problem as a partially observable Markov decision process (POMDP) and model the environment as the codebook itself. At each time step, we select a candidate beam conditioned on the belief state of the unobservable optimal beam and previously probed beams. This frames the beam selection problem as an online search procedure that locates the moving optimal beam. In contrast to previous work, our method handles new or unforeseen trajectories and changes in the physical environment, and outperforms previous work by orders of magnitude.