# Optimal Control of Fluid Restless Multi-armed Bandits: A Machine Learning Approach

arXiv: https://arxiv.org/abs/2502.03725 · HTML: https://mlsanity.com/paper/2502.03725/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 59 references that are in the map.
- First posted on arXiv 6 Feb 2025.

## Builds on (most cited first, 8 of 8)

- [Field Study in Deploying Restless Multi-Armed Bandits: Assisting Non-Profits in Improving Maternal and Child Health](https://mlsanity.com/paper/2109.08075/) (2021)
- [Restless Bandits with Many Arms: Beating the Central Limit Theorem](https://mlsanity.com/paper/2107.11911/) (2021)
- [LP-based policies for restless bandits: necessary and sufficient conditions for (exponentially fast) asymptotic optimality](https://mlsanity.com/paper/2106.10067/) (2021)
- [Hamilton-Jacobi-Bellman Equations for Q-Learning in Continuous Time](https://mlsanity.com/paper/1912.10697/) (2019)
- [Restless Bandits with Average Reward: Breaking the Uniform Global Attractor Assumption](https://mlsanity.com/paper/2306.00196/) (2023)
- [Continuous-Time Fitted Value Iteration for Robust Policies](https://mlsanity.com/paper/2110.01954/) (2021)
- [An Optimal-Control Approach to Infinite-Horizon Restless Bandits: Achieving Asymptotic Optimality with Minimal Assumptions](https://mlsanity.com/paper/2403.11913/) (2024)
- [Optimal Control of Multiclass Fluid Queueing Networks: A Machine Learning Approach](https://mlsanity.com/paper/2307.12405/) (2023)

## Abstract (arXiv)

We present a novel machine learning framework for the optimal control of fluid restless multi-armed bandit problems (FRMABPs) with state equations that are either affine or quadratic in the state variables. By establishing fundamental properties of FRMABPs, we develop an efficient numerical algorithm that generates a comprehensive training set by solving multiple instances with diverse initial states. We further enhance this training set by applying a nonlinear transformation to the feature vectors, leveraging structural properties of FRMABPs. A time-dependent state feedback policy is then learned using Optimal Classification Trees with hyperplane splits (OCT-H). We test our approach on machine maintenance, epidemic control, and fisheries control problems, demonstrating that our method yields high-quality state feedback policies. Furthermore, once a policy is learned, it achieves a speed-up of up to 26 million times compared to the direct numerical algorithm.