# Optimal Epidemic Control as a Contextual Combinatorial Bandit with Budget

arXiv: https://arxiv.org/abs/2106.15808 · HTML: https://mlsanity.com/paper/2106.15808/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 29 references that are in the map.
- First posted on arXiv 30 Jun 2021.

## Builds on (most cited first, 9 of 9)

- [A Survey on Practical Applications of Multi-Armed and Contextual Bandits](https://mlsanity.com/paper/1904.10040/) (2019)
- [A Neural Networks Committee for the Contextual Bandit Problem](https://mlsanity.com/paper/1409.8191/) (2014)
- [Multi-armed Bandit Problem with Known Trend](https://mlsanity.com/paper/1508.07091/) (2015)
- [Incorporating Behavioral Constraints in Online AI Systems](https://mlsanity.com/paper/1809.05720/) (2018)
- [Context Attentive Bandits: Contextual Bandit with Restricted Context](https://mlsanity.com/paper/1705.03821/) (2017)
- [Unified Models of Human Behavioral Agents in Bandits, Contextual Bandits and RL](https://mlsanity.com/paper/2005.04544/) (2020)
- [Speaker Diarization as a Fully Online Learning Problem in MiniVox](https://mlsanity.com/paper/2006.04376/) (2020)
- [Online Learning in Iterated Prisoner's Dilemma to Mimic Human Behavior](https://mlsanity.com/paper/2006.06580/) (2020)
- [Online Semi-Supervised Learning in Contextual Bandits with Episodic Reward](https://mlsanity.com/paper/2009.08457/) (2020)

## Built on by (most cited first, 5 of 5)

- [Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlook](https://mlsanity.com/paper/2210.13623/) (2022)
- [Survey: Multi-Armed Bandits Meet Large Language Models](https://mlsanity.com/paper/2505.13355/) (2025)
- [Offline Learning for Combinatorial Multi-armed Bandits](https://mlsanity.com/paper/2501.19300/) (2025)
- [Evolutionary Multi-Armed Bandits with Genetic Thompson Sampling](https://mlsanity.com/paper/2205.10113/) (2022)
- [Bayesian Analysis of Combinatorial Gaussian Process Bandits](https://mlsanity.com/paper/2312.12676/) (2023)

## Abstract (arXiv)

In light of the COVID-19 pandemic, it is an open challenge and critical practical problem to find a optimal way to dynamically prescribe the best policies that balance both the governmental resources and epidemic control in different countries and regions. To solve this multi-dimensional tradeoff of exploitation and exploration, we formulate this technical challenge as a contextual combinatorial bandit problem that jointly optimizes a multi-criteria reward function. Given the historical daily cases in a region and the past intervention plans in place, the agent should generate useful intervention plans that policy makers can implement in real time to minimizing both the number of daily COVID-19 cases and the stringency of the recommended interventions. We prove this concept with simulations of multiple realistic policy making scenarios and demonstrate a clear advantage in providing a pareto optimal solution in the epidemic intervention problem.