# Fast Change Identification in Multi-Play Bandits and its Applications in Wireless Networks

arXiv: https://arxiv.org/abs/2205.10366 · HTML: https://mlsanity.com/paper/2205.10366/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 45 references that are in the map.
- First posted on arXiv 20 May 2022.

## Builds on (most cited first, 16 of 16)

- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Parallel Gaussian Process Optimization with Upper Confidence Bound and Pure Exploration](https://mlsanity.com/paper/1304.5350/) (2013)
- [Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Bandit](https://mlsanity.com/paper/1802.03692/) (2018)
- [Taming Non-stationary Bandits: A Bayesian Approach](https://mlsanity.com/paper/1707.09727/) (2017)
- [Reinforcement Learning for IoT Security: A Comprehensive Survey](https://mlsanity.com/paper/2102.07247/) (2021)
- [Non-Stationary Bandit Learning via Predictive Sampling](https://mlsanity.com/paper/2205.01970/) (2022)
- [Joint Status Sampling and Updating for Minimizing Age of Information in the Internet of Things](https://mlsanity.com/paper/1807.04356/) (2018)
- [A Change-Detection Based Thompson Sampling Framework for Non-Stationary Bandits](https://mlsanity.com/paper/2009.02791/) (2020)
- [Coverage and Rate Analysis for Millimeter Wave Cellular Networks](https://mlsanity.com/paper/1402.6430/) (2014)
- [Collaborative Multi-Agent Multi-Armed Bandit Learning for Small-Cell Caching](https://mlsanity.com/paper/2001.03835/) (2020)
- [Multi-player Multi-armed Bandits with Collision-Dependent Reward Distributions](https://mlsanity.com/paper/2106.13669/) (2021)
- [Cost-aware Cascading Bandits](https://mlsanity.com/paper/1805.08638/) (2018)
- [Resource Allocation in NOMA-based Self-Organizing Networks using Stochastic Multi-Armed Bandits](https://mlsanity.com/paper/2101.06340/) (2021)
- [User Association in Dense mmWave Networks as Restless Bandits](https://mlsanity.com/paper/2107.09153/) (2021)
- [Kolmogorov-Smirnov Test-Based Actively-Adaptive Thompson Sampling for Non-Stationary Bandits](https://mlsanity.com/paper/2105.14586/) (2021)
- [An Online Algorithm for Computation Offloading in Non-Stationary Environments](https://mlsanity.com/paper/2006.12032/) (2020)

## Built on by (most cited first, 2 of 2)

- [Online Optimization for Learning to Communicate over Time-Correlated Channels](https://mlsanity.com/paper/2409.00575/) (2024)
- [An Algorithm for Fixed Budget Best Arm Identification with Combinatorial Exploration](https://mlsanity.com/paper/2502.01429/) (2025)

## Abstract (arXiv)

Next-generation wireless services are characterized by a diverse set of requirements, to sustain which, the wireless access points need to probe the users in the network periodically. In this regard, we study a novel multi-armed bandit (MAB) setting that mandates probing all the arms periodically while keeping track of the best current arm in a non-stationary environment. In particular, we develop textttTS-GE that balances the regret guarantees of classical Thompson sampling (TS) with the broadcast probing (BP) of all the arms simultaneously in order to actively detect a change in the reward distributions. The main innovation in the algorithm is in identifying the changed arm by an optional subroutine called group exploration (GE) that scales as log2(K) for a K-armed bandit setting. We characterize the probability of missed detection and the probability of false-alarm in terms of the environment parameters. We highlight the conditions in which the regret guarantee of textttTS-GE outperforms that of the state-of-the-art algorithms, in particular, textttADSWITCH and textttM-UCB. We demonstrate the efficacy of textttTS-GE by employing it in two wireless system application - task offloading in mobile-edge computing (MEC) and an industrial internet-of-things (IIoT) network designed for simultaneous wireless information and power transfer (SWIPT).