# An Analysis of Reinforcement Learning for Malaria Control

arXiv: https://arxiv.org/abs/2107.08988 · HTML: https://mlsanity.com/paper/2107.08988/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 49 references that are in the map.
- First posted on arXiv 19 Jul 2021.

## Builds on (most cited first, 17 of 17)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Reinforcement Learning in Healthcare: A Survey](https://mlsanity.com/paper/1908.08796/) (2019)
- [Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation](https://mlsanity.com/paper/1807.01473/) (2018)
- [Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1705.08422/) (2017)
- [Algorithms for multi-armed bandit problems](https://mlsanity.com/paper/1402.6028/) (2014)
- [Evaluating Reinforcement Learning Algorithms in Observational Health Settings](https://mlsanity.com/paper/1805.12298/) (2018)
- [Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adapting](https://mlsanity.com/paper/1902.01520/) (2019)
- [Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data](https://mlsanity.com/paper/1801.09271/) (2018)
- [Deep reinforcement learning for large-scale epidemic control](https://mlsanity.com/paper/2003.13676/) (2020)
- [Representation and Reinforcement Learning for Personalized Glycemic Control in Septic Patients](https://mlsanity.com/paper/1712.00654/) (2017)
- [EpidemiOptim: A Toolbox for the Optimization of Control Policies in Epidemiological Models](https://mlsanity.com/paper/2010.04452/) (2020)
- [Adaptive Rate of Convergence of Thompson Sampling for Gaussian Process Optimization](https://mlsanity.com/paper/1705.06808/) (2017)
- [Artificial Intelligence Framework for Simulating Clinical Decision-Making: A Markov Decision Process Approach](https://mlsanity.com/paper/1301.2158/) (2013)
- [Optimization of anemia treatment in hemodialysis patients via reinforcement learning](https://mlsanity.com/paper/1509.03977/) (2015)
- [Precision medicine as a control problem: Using simulation and deep reinforcement learning to discover adaptive, personalized multi-cytokine therapy for sepsis](https://mlsanity.com/paper/1802.10440/) (2018)
- [Novel Exploration Techniques (NETs) for Malaria Policy Interventions](https://mlsanity.com/paper/1712.00428/) (2017)
- [Policy Learning for Malaria Control](https://mlsanity.com/paper/1910.08926/) (2019)

## Abstract (arXiv)

Previous work on policy learning for Malaria control has often formulated the problem as an optimization problem assuming the objective function and the search space have a specific structure. The problem has been formulated as multi-armed bandits, contextual bandits and a Markov Decision Process in isolation. Furthermore, an emphasis is put on developing new algorithms specific to an instance of Malaria control, while ignoring a plethora of simpler and general algorithms in the literature. In this work, we formally study the formulation of Malaria control and present a comprehensive analysis of several formulations used in the literature. In addition, we implement and analyze several reinforcement learning algorithms in all formulations and compare them to black box optimization. In contrast to previous work, our results show that simple algorithms based on Upper Confidence Bounds are sufficient for learning good Malaria policies, and tend to outperform their more advanced counterparts on the malaria OpenAI Gym environment.