# Dynamic priority allocation via restless bandit marginal productivity indices

arXiv: https://arxiv.org/abs/2304.06115 · HTML: https://mlsanity.com/paper/2304.06115/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2023.

## Built on by (most cited first, 8 of 8)

- [Online Restless Multi-Armed Bandits with Long-Term Fairness Constraints](https://mlsanity.com/paper/2312.10303/) (2023)
- [Indexability of Finite State Restless Multi-Armed Bandit and Rollout Policy](https://mlsanity.com/paper/2305.00410/) (2023)
- [General Formulation and PCL-Analysis for Restless Bandits with Limited Observability](https://mlsanity.com/paper/2307.03034/) (2023)
- [GINO-Q: Learning an Asymptotically Optimal Index Policy for Restless Multi-armed Bandits](https://mlsanity.com/paper/2408.09882/) (2024)
- [A Novel Switch-Type Policy Network for Resource Allocation Problems: Technical Report](https://mlsanity.com/paper/2501.11136/) (2025)
- [Structured Reinforcement Learning for Delay-Optimal Data Transmission in Dense mmWave Networks](https://mlsanity.com/paper/2404.16920/) (2024)
- [Risk-Aware Decision Making in Restless Bandits: Theory and Algorithms for Planning and Learning](https://mlsanity.com/paper/2410.23029/) (2024)
- [From Relaxed Indexability to Exact Indexability: A t-Step Approach for Partially Observable Restless Bandits](https://mlsanity.com/paper/2608.24167/) (2026)

## Abstract (arXiv)

This paper surveys recent work by the author on the theoretical and algorithmic aspects of restless bandit indexation as well as on its application to a variety of problems involving the dynamic allocation of priority to multiple stochastic projects. The main aim is to present ideas and methods in an accessible form that can be of use to researchers addressing problems of such a kind. Besides building on the rich literature on bandit problems, our approach draws on ideas from linear programming, economics, and multi-objective optimization. In particular, it was motivated to address issues raised in the seminal work of Whittle (Restless bandits: activity allocation in a changing world. In: Gani J. (ed.) A Celebration of Applied Probability, J. Appl. Probab., vol. 25A, Applied Probability Trust, Sheffield, pp. 287-298, 1988) where he introduced the index for restless bandits that is the starting point of this work. Such an index, along with previously proposed indices and more recent extensions, is shown to be unified through the intuitive concept of ``marginal productivity index'' (MPI), which measures the marginal productivity of work on a project at each of its states. In a multi-project setting, MPI policies are economically sound, as they dynamically allocate higher priority to those projects where work appears to be currently more productive. Besides being tractable and widely applicable, a growing body of computational evidence indicates that such index policies typically achieve a near-optimal performance and substantially outperform benchmark policies derived from conventional approaches.