mlsanityData to 7 Oct 2026

NeurWIN: Neural Whittle Index Network For Restless Bandits Via Deep RL

arXiv 2110.02128 ↗First posted 5 Oct 2021cs.LG stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

Whittle index policy is a powerful tool to obtain asymptotically optimal solutions for the notoriously intractable problem of restless bandits. However, finding the Whittle indices remains a difficult problem for many practical restless bandits with convoluted transition kernels. This paper proposes NeurWIN, a neural Whittle index network that seeks to learn the Whittle indices for any restless bandits by leveraging mathematical properties of the Whittle indices. We show that a neural network that produces the Whittle index is also one that produces the optimal control for a set of Markov decision problems. This property motivates using deep reinforcement learning for the training of NeurWIN. We demonstrate the utility of NeurWIN by evaluating its performance for three recently studied restless bandit problems. Our experiment results show that the performance of NeurWIN is significantly better than other RL algorithms.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 19 of its 38 references that are in the map. First posted on arXiv 5 Oct 2021.

Builds on

19 of 38 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  4. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  5. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
  6. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  7. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  8. 2015Deep Reinforcement Learning in Large Discrete Action Spacescited by 158
  9. 2020Whittle index based Q-learning for restless bandits with average rewardcited by 43
  10. 2018Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networkscited by 34
  11. 2021Learn to Intervene: An Adaptive Learning Policy for Restless Bandits in Application to Preventive Healthcarecited by 34
  12. 2020Q-Learning in enormous action spaces via amortized approximate maximizationcited by 30
7 more
  1. 2019Recovering Banditscited by 28
  2. 2018Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systemscited by 21
  3. 2019A Whittle Index Approach to Minimizing Functions of Age of Informationcited by 20
  4. 2016Deadline Scheduling as Restless Banditscited by 16
  5. 2016On the Whittle Index for Restless Multi-armed Hidden Markov Banditscited by 12
  6. 2019Problem Dependent Reinforcement Learning Bounds Which Can Identify Bandit Structure in MDPscited by 6
  7. 2015When are Kalman-filter restless bandits indexable?cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 11; 2025: 7; 2026: 1.

27 papers.

Sort:

By area:

  1. 2024A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Healthcited by 24
  2. 2022Optimistic Whittle Index Policy: Online Learning for Restless Banditscited by 18
  3. 2023Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximationcited by 14
  4. 2023Online Restless Multi-Armed Bandits with Long-Term Fairness Constraintscited by 10
  5. 2022Near-optimality for infinite-horizon restless bandits with many armscited by 9
  6. 2023Towards a Pretrained Model for Restless Bandits via Multi-arm Generalizationcited by 9
  7. 2022DeepTOP: Deep Threshold-Optimal Policy for MDPs and RMABscited by 6
  8. 2023Weakly Coupled Deep Q-Networkscited by 6
  9. 2023Full Gradient Deep Reinforcement Learning for Average-Reward Criterioncited by 5
  10. 2024The Bandit Whisperer: Communication Learning for Restless Banditscited by 5
  11. 2025Reinforcement learning with combinatorial actions for coupled restless banditscited by 5
  12. 2024Tabular and Deep Learning for the Whittle Indexcited by 3
15 more
  1. 2024Faster Q-Learning Algorithms for Restless Banditscited by 2
  2. 2024DOPL: Direct Online Preference Learning for Restless Bandits with Preference Feedbackcited by 2
  3. 2024Lagrangian Index Policy for Restless Bandits with Average Rewardcited by 2
  4. 2025Projection-based Lyapunov method for fully heterogeneous weakly-coupled MDPscited by 2
  5. 2024Deep reinforcement learning for weakly coupled MDP's with continuous actionscited by 1
  6. 2024GINO-Q: Learning an Asymptotically Optimal Index Policy for Restless Multi-armed Banditscited by 1
  7. 2024Whittle Index Learning Algorithms for Restless Bandits with Constant Stepsizescited by 1
  8. 2024ContextWIN: Whittle Index Based Mixture-of-Experts Neural Model For Restless Bandits Via Deep RLcited by 1
  9. 2025Finite-Horizon Single-Pull Restless Bandits: An Efficient Index Policy For Scarce Resource Allocationcited by 1
  10. 2025Online Learning of Whittle Indices for Restless Bandits with Non-Stationary Transition Kernelscited by 1
  11. 2024An Index Policy Based on Sarsa and Q-learning for Heterogeneous Smart Target Trackingcited by 0
  12. 2025Multi-agent Markov Entanglementcited by 0
  13. 2025MARBLE: Multi-Armed Restless Bandits in Latent Markovian Environmentcited by 0
  14. 2025Model-Based Learning of Whittle indicescited by 0
  15. 2026Optimizing Resource-Constrained Non-Pharmaceutical Interventions for Multi-Cluster Outbreak Control Using Hierarchical Reinforcement Learningcited by 0