# Two families of indexable partially observable restless bandits and Whittle index computation

arXiv: https://arxiv.org/abs/2104.05151 · HTML: https://mlsanity.com/paper/2104.05151/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2021.

## Built on by (most cited first, 2 of 2)

- [On learning Whittle index policy for restless bandits with scalable regret](https://mlsanity.com/paper/2202.03463/) (2022)
- [Threshold Structure of Optimal Policies in Restart POMDPs](https://mlsanity.com/paper/2608.10936/) (2026)

## Abstract (arXiv)

We consider the restless bandits with general state space under partial observability with two observational models: first, the state of each bandit is not observable at all, and second, the state of each bandit is observable only if it is chosen. We assume both models satisfy the restart property under which we prove indexability of the models and propose the Whittle index policy as the solution. For the first model, we derive a closed-form expression for the Whittle index. For the second model, we propose an efficient algorithm to compute the Whittle index by exploiting the qualitative properties of the optimal policy. We present detailed numerical experiments for multiple instances of machine maintenance problem. The result indicates that the Whittle index policy outperforms myopic policy and can be close to optimal in different setups.