# Adaptive Scheduling: A Reinforcement Learning Whittle Index Approach for Wireless Sensor Networks

arXiv: https://arxiv.org/abs/2601.01179 · HTML: https://mlsanity.com/paper/2601.01179/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 55 references that are in the map.
- First posted on arXiv 3 Jan 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 19 of 19)

- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Combinatorial Multi-Armed Bandit and Its Extension to Probabilistically Triggered Arms](https://mlsanity.com/paper/1407.8339/) (2014)
- [Whittle index based Q-learning for restless bandits with average reward](https://mlsanity.com/paper/2004.14427/) (2020)
- [Learn to Intervene: An Adaptive Learning Policy for Restless Bandits in Application to Preventive Healthcare](https://mlsanity.com/paper/2105.07965/) (2021)
- [Restless-UCB, an Efficient and Low-complexity Algorithm for Online Restless Bandits](https://mlsanity.com/paper/2011.02664/) (2020)
- [Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems](https://mlsanity.com/paper/1809.05149/) (2018)
- [Optimistic Whittle Index Policy: Online Learning for Restless Bandits](https://mlsanity.com/paper/2205.15372/) (2022)
- [Goal-Oriented Communications for the IoT and Application to Data Compression](https://mlsanity.com/paper/2211.05378/) (2022)
- [Making Sense of Meaning: A Survey on Metrics for Semantic and Goal-Oriented Communication](https://mlsanity.com/paper/2303.10867/) (2023)
- [DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling](https://mlsanity.com/paper/1803.02998/) (2018)
- [Remote State Estimation with Smart Sensors over Markov Fading Channels](https://mlsanity.com/paper/2005.07871/) (2020)
- [Whittle Index based Q-Learning for Wireless Edge Caching with Linear Function Approximation](https://mlsanity.com/paper/2202.13187/) (2022)
- [Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Bandits](https://mlsanity.com/paper/2402.04933/) (2024)
- [Push- and Pull-based Effective Communication in Cyber-Physical Systems](https://mlsanity.com/paper/2401.10921/) (2024)
- [Optimizing AoI at Query in Multiuser Wireless Uplink Networks: A Whittle Index Approach](https://mlsanity.com/paper/2411.02108/) (2024)
- [Robbins-Monro conditions for persistent exploration learning strategies](https://mlsanity.com/paper/1808.00245/) (2018)
- [Combinatorial-restless-bandit-based Transmitter-Receiver Online Selection for Distributed MIMO Radars With Non-Stationary Channels](https://mlsanity.com/paper/2306.09710/) (2023)
- [Energy-Efficient Internet of Things Monitoring with Content-Based Wake-Up Radio](https://mlsanity.com/paper/2312.04294/) (2023)
- [Adaptive Client Selection via Q-Learning-based Whittle Index in Wireless Federated Learning](https://mlsanity.com/paper/2509.13933/) (2025)

## Abstract (arXiv)

We propose a reinforcement learning based scheduling framework for Restless Multi-Armed Bandit (RMAB) problems, centred on a Whittle Index Q-Learning policy with Upper Confidence Bound (UCB) exploration, referred to as WIQL-UCB. Unlike existing approaches that rely on fixed or adaptive epsilon-greedy strategies and require careful hyperparameter tuning, the proposed method removes problem-specific tuning and is therefore more generalisable across diverse RMAB settings. We evaluate WIQL-UCB on standard RMAB benchmarks and on a practical sensor scheduling application based on the Age of Incorrect Information (AoII), using an edge-based state estimation scheme that requires no prior knowledge of system dynamics. Experimental results show that WIQL-UCB achieves near-optimal performance while significantly improving computational and memory efficiency. For a representative problem size of N = 15 and M = 3, the proposed method requires only around 600 bytes of memory, compared with several kilobytes for tabular Q-learning and hundreds of kilobytes to megabytes for deep reinforcement learning baselines. In addition, WIQL-UCB achieves sub-millisecond per-decision runtimes and is several times faster than deep reinforcement learning approaches, while maintaining competitive performance. Overall, these results demonstrate that WIQL-UCB consistently outperforms both non-Whittle-based and Whittle-index learning baselines across a wide range of RMAB settings.