# A Restless Bandit Model for Resource Allocation, Competition and Reservation

arXiv: https://arxiv.org/abs/1804.02100 · HTML: https://mlsanity.com/paper/1804.02100/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2018.

## Built on by (most cited first, 2 of 2)

- [An Index Policy Based on Sarsa and Q-learning for Heterogeneous Smart Target Tracking](https://mlsanity.com/paper/2402.12015/) (2024)
- [Bandits in Flux: Adversarial Constraints in Dynamic Environments](https://mlsanity.com/paper/2601.19867/) (2026)

## Abstract (arXiv)

We study a resource allocation problem with varying requests, and with resources of limited capacity shared by multiple requests. It is modeled as a set of heterogeneous Restless Multi-Armed Bandit Problems (RMABPs) connected by constraints imposed by resource capacity. Following Whittle's relaxation idea and Weber and Weiss' asymptotic optimality proof, we propose a simple policy and prove it to be asymptotically optimal in a regime where both arrival rates and capacities increase. We provide a simple sufficient condition for asymptotic optimality of the policy, and in complete generality propose a method that generates a set of candidate policies for which asymptotic optimality can be checked. The effectiveness of these results is demonstrated by numerical experiments. To the best of our knowledge, this is the first work providing asymptotic optimality results for such a resource allocation problem and such a combination of multiple RMABPs.