Adaptive Scheduling: A Reinforcement Learning Whittle Index Approach for Wireless Sensor Networks
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
We propose a reinforcement learning based scheduling framework for Restless Multi-Armed Bandit (RMAB) problems, centred on a Whittle Index Q-Learning policy with Upper Confidence Bound (UCB) exploration, referred to as WIQL-UCB. Unlike existing approaches that rely on fixed or adaptive epsilon-greedy strategies and require careful hyperparameter tuning, the proposed method removes problem-specific tuning and is therefore more generalisable across diverse RMAB settings. We evaluate WIQL-UCB on standard RMAB benchmarks and on a practical sensor scheduling application based on the Age of Incorrect Information (AoII), using an edge-based state estimation scheme that requires no prior knowledge of system dynamics. Experimental results show that WIQL-UCB achieves near-optimal performance while significantly improving computational and memory efficiency. For a representative problem size of N = 15 and M = 3, the proposed method requires only around 600 bytes of memory, compared with several kilobytes for tabular Q-learning and hundreds of kilobytes to megabytes for deep reinforcement learning baselines. In addition, WIQL-UCB achieves sub-millisecond per-decision runtimes and is several times faster than deep reinforcement learning approaches, while maintaining competitive performance. Overall, these results demonstrate that WIQL-UCB consistently outperforms both non-Whittle-based and Whittle-index learning baselines across a wide range of RMAB settings.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 19 of its 55 references that are in the map. First posted on arXiv 3 Jan 2026.
Builds on
19 of 55 references.
References · 19
- 2022Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communicationscited by 175
- 2014Combinatorial Multi-Armed Bandit and Its Extension to Probabilistically Triggered Armscited by 79
- 2020Whittle index based Q-learning for restless bandits with average rewardcited by 45
- 2021Learn to Intervene: An Adaptive Learning Policy for Restless Bandits in Application to Preventive Healthcarecited by 35
- 2020Restless-UCB, an Efficient and Low-complexity Algorithm for Online Restless Banditscited by 27
- 2018Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systemscited by 23
- 2022Optimistic Whittle Index Policy: Online Learning for Restless Banditscited by 20
- 2022Goal-Oriented Communications for the IoT and Application to Data Compressioncited by 12
- 2023Making Sense of Meaning: A Survey on Metrics for Semantic and Goal-Oriented Communicationcited by 10
- 2018DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Schedulingcited by 9
- 2020Remote State Estimation with Smart Sensors over Markov Fading Channelscited by 8
- 2022Whittle Index based Q-Learning for Wireless Edge Caching with Linear Function Approximationcited by 8
7 more
- 2024Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Banditscited by 6
- 2024Push- and Pull-based Effective Communication in Cyber-Physical Systemscited by 2
- 2024Optimizing AoI at Query in Multiuser Wireless Uplink Networks: A Whittle Index Approachcited by 2
- 2018Robbins-Monro conditions for persistent exploration learning strategiescited by 1
- 2023Combinatorial-restless-bandit-based Transmitter-Receiver Online Selection for Distributed MIMO Radars With Non-Stationary Channelscited by 1
- 2023Energy-Efficient Internet of Things Monitoring with Content-Based Wake-Up Radiocited by 1
- 2025Adaptive Client Selection via Q-Learning-based Whittle Index in Wireless Federated Learningcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Restless Bandits with Individual Penalty Constraints: Near-Optimal Indices and Deep Reinforcement Learning
- 2025Multi-Task Lifelong Reinforcement Learning for Wireless Sensor Networks
- 2024Deep Index Policy for Multi-Resource Restless Matching Bandit and Its Application in Multi-Channel Scheduling
- 2024HGFF: A Deep Reinforcement Learning Framework for Lifetime Maximization in Wireless Sensor Networks
- 2025Enhanced Evolutionary Multi-Objective Deep Reinforcement Learning for Reliable and Efficient Wireless Rechargeable Sensor Networks
- 2026HQARRF: Hierarchical Q-learning and Force-aware Routing for Multi-Charger Scheduling in Wireless Rechargeable Sensor Networks
- 2022On learning Whittle index policy for restless bandits with scalable regret
- 2023Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets