# Contextual Bandits with Non-Stationary Correlated Rewards for User Association in MmWave Vehicular Networks

arXiv: https://arxiv.org/abs/2410.05785 · HTML: https://mlsanity.com/paper/2410.05785/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 41 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 11 of 11)

- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Online Learning: A Modern Introduction Using Convex Optimization](https://mlsanity.com/paper/1912.13213/) (2019)
- [Multi-Armed Bandits with Correlated Arms](https://mlsanity.com/paper/1911.03959/) (2019)
- [Millimeter Wave Channel Modeling and Cellular Capacity Evaluation](https://mlsanity.com/paper/1312.4921/) (2013)
- [User Association in 5G Networks: A Survey and an Outlook](https://mlsanity.com/paper/1509.00338/) (2015)
- [Joint Channel Selection and Power Control in Infrastructureless Wireless Networks: A Multi-Player Multi-Armed Bandit Framework](https://mlsanity.com/paper/1407.5447/) (2014)
- [Channel Estimation for RIS-Aided Multiuser Millimeter-Wave Systems](https://mlsanity.com/paper/2106.14792/) (2021)
- [Millimeter Wave Cellular Networks: A MAC Layer Perspective](https://mlsanity.com/paper/1503.00697/) (2015)
- [Propagation Channels for mmWave Vehicular Communications: State-of-the-art and Future Research Directions](https://mlsanity.com/paper/1812.02483/) (2018)
- [Learning-based Load Balancing Handover in Mobile Millimeter Wave Networks](https://mlsanity.com/paper/2011.01420/) (2020)
- [Mobility-Aware Resource Allocation for mmWave IAB Networks: A Multi-Agent Reinforcement Learning Approach](https://mlsanity.com/paper/2205.06011/) (2022)

## Built on by (most cited first, 2 of 2)

- [Learning-Based User Association for MmWave Vehicular Networks With Kernelized Contextual Bandits](https://mlsanity.com/paper/2504.10959/) (2025)
- [Beam-aware Kernelized Contextual Bandits for User Association and Beamforming in mmWave Vehicular Networks](https://mlsanity.com/paper/2603.19285/) (2026)

## Abstract (arXiv)

Millimeter wave (mmWave) communication has emerged as a propelling technology in vehicular communication. Usually, an appropriate decision on user association requires timely channel information between vehicles and base stations (BSs), which is challenging given a fast-fading mmWave vehicular channel. In this paper, relying solely on learning transmission rate, we propose a low-complexity semi-distributed contextual correlated upper confidence bound (SD-CC-UCB) algorithm to establish an up-to-date user association without explicit measurement of channel state information (CSI). Under a contextual multi-arm bandits framework, SD-CC-UCB learns and predicts the transmission rate given the location and velocity of the vehicle, which can adequately capture the intricate channel condition for a prompt decision on user association. Further, SD-CC-UCB efficiently identifies the set of candidate BSs which probably support supreme transmission rate by leveraging the correlated distributions of transmission rates on different locations. To further refine the learning transmission rate over the link to candidate BSs, each vehicle deploys the Thompson Sampling algorithm by taking the interference among vehicles and handover overhead into consideration. Numerical results show that our proposed algorithm achieves the network throughput within 100%-103% of a benchmark algorithm which requires perfect instantaneous CSI, demonstrating the effectiveness of SD-CC-UCB in vehicular communications.