mlsanityData to 7 Oct 2026

Whittle index based Q-learning for restless bandits with average reward

arXiv 2004.14427 ↗First posted 29 Apr 2020cs.LG math.OC stat.MLcited by 43

Not on the map. Why?

Abstract (arXiv)

A novel reinforcement learning algorithm is introduced for multiarmed restless bandits with average reward, using the paradigms of Q-learning and Whittle index. Specifically, we leverage the structure of the Whittle index policy to reduce the search space of Q-learning, resulting in major computational gains. Rigorous convergence analysis is provided, supported by numerical experiments. The numerical experiments show excellent empirical performance of the proposed scheme.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 7; 2022: 6; 2023: 7; 2024: 14; 2025: 7; 2026: 1.

43 papers.

Sort:

By area:

  1. 2021Field Study in Deploying Restless Multi-Armed Bandits: Assisting Non-Profits in Improving Maternal and Child Healthcited by 43
  2. 2021Learn to Intervene: An Adaptive Learning Policy for Restless Bandits in Application to Preventive Healthcarecited by 34
  3. 2021NeurWIN: Neural Whittle Index Network For Restless Bandits Via Deep RLcited by 27
  4. 2021Q-Learning Lagrange Policies for Multi-Action Restless Banditscited by 20
  5. 2020Average-reward model-free reinforcement learning: a systematic review and literature mappingcited by 19
  6. 2022Optimistic Whittle Index Policy: Online Learning for Restless Banditscited by 18
  7. 2023Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximationcited by 14
  8. 2022Efficient Resource Allocation with Fairness Constraints in Restless Multi-Armed Banditscited by 11
  9. 2023Online Restless Multi-Armed Bandits with Long-Term Fairness Constraintscited by 10
  10. 2022On learning Whittle index policy for restless bandits with scalable regretcited by 9
  11. 2023Policy Optimization for Personalized Interventions in Behavioral Healthcited by 9
  12. 2023Towards a Pretrained Model for Restless Bandits via Multi-arm Generalizationcited by 9
31 more
  1. 2021Restless and Uncertain: Robust Policies for Restless Bandits via Deep Multi-Agent Reinforcement Learningcited by 7
  2. 2022Whittle Index based Q-Learning for Wireless Edge Caching with Linear Function Approximationcited by 7
  3. 2022DeepTOP: Deep Threshold-Optimal Policy for MDPs and RMABscited by 6
  4. 2023Weakly Coupled Deep Q-Networkscited by 6
  5. 2023Full Gradient Deep Reinforcement Learning for Average-Reward Criterioncited by 5
  6. 2024The Bandit Whisperer: Communication Learning for Restless Banditscited by 5
  7. 2025Reinforcement learning with combinatorial actions for coupled restless banditscited by 5
  8. 2021Learning Augmented Index Policy for Optimal Service Placement at the Network Edgecited by 4
  9. 2024Tabular and Deep Learning for the Whittle Indexcited by 3
  10. 2024Faster Q-Learning Algorithms for Restless Banditscited by 2
  11. 2024DOPL: Direct Online Preference Learning for Restless Bandits with Preference Feedbackcited by 2
  12. 2024Lagrangian Index Policy for Restless Bandits with Average Rewardcited by 2
  13. 2025Projection-based Lyapunov method for fully heterogeneous weakly-coupled MDPscited by 2
  14. 2021Reinforcement Learning for Markovian Bandits: Is Posterior Sampling more Scalable than Optimism?cited by 1
  15. 2023Indexability of Finite State Restless Multi-Armed Bandit and Rollout Policycited by 1
  16. 2024Provably Efficient Reinforcement Learning for Adversarial Restless Multi-Armed Bandits with Unknown Transitions and Bandit Feedbackcited by 1
  17. 2024Tabular and Deep Reinforcement Learning for Gittins Indexcited by 1
  18. 2024Deep reinforcement learning for weakly coupled MDP's with continuous actionscited by 1
  19. 2024GINO-Q: Learning an Asymptotically Optimal Index Policy for Restless Multi-armed Banditscited by 1
  20. 2024Whittle Index Learning Algorithms for Restless Bandits with Constant Stepsizescited by 1
  21. 2024Model Predictive Control is Almost Optimal for Restless Banditcited by 1
  22. 2025Online Learning of Whittle Indices for Restless Bandits with Non-Stationary Transition Kernelscited by 1
  23. 2025Model Predictive Control is almost Optimal for Heterogeneous Restless Multi-armed Banditscited by 1
  24. 2022Caching Contents with Varying Popularity using Restless Banditscited by 0
  25. 2024An Index Policy Based on Sarsa and Q-learning for Heterogeneous Smart Target Trackingcited by 0
  26. 2024A resource-constrained stochastic scheduling algorithm for homeless street outreach and gleaning edible foodcited by 0
  27. 2024Structured Reinforcement Learning for Delay-Optimal Data Transmission in Dense mmWave Networkscited by 0
  28. 2025Multi-agent Markov Entanglementcited by 0
  29. 2025MARBLE: Multi-Armed Restless Bandits in Latent Markovian Environmentcited by 0
  30. 2025Model-Based Learning of Whittle indicescited by 0
  31. 2026Optimizing Resource-Constrained Non-Pharmaceutical Interventions for Multi-Cluster Outbreak Control Using Hierarchical Reinforcement Learningcited by 0