# ESCADA: Efficient Safety and Context Aware Dose Allocation for Precision Medicine

arXiv: https://arxiv.org/abs/2111.13415 · HTML: https://mlsanity.com/paper/2111.13415/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 71 references that are in the map.
- First posted on arXiv 26 Nov 2021.

## Builds on (most cited first, 25 of 25)

- [On Information Gain and Regret Bounds in Gaussian Process Bandits](https://mlsanity.com/paper/2009.06966/) (2020)
- [Ethical Machine Learning in Health Care](https://mlsanity.com/paper/2009.10576/) (2020)
- [Bayesian Optimization with Unknown Constraints](https://mlsanity.com/paper/1403.5607/) (2014)
- [Stagewise Safe Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1806.07555/) (2018)
- [Conservative Bandits](https://mlsanity.com/paper/1602.04282/) (2016)
- [Conservative Contextual Linear Bandits](https://mlsanity.com/paper/1611.06426/) (2016)
- [Safe Exploration for Interactive Machine Learning](https://mlsanity.com/paper/1910.13726/) (2019)
- [A General Framework for Constrained Bayesian Optimization using Information-based Search](https://mlsanity.com/paper/1511.09422/) (2015)
- [Universal Off-Policy Evaluation](https://mlsanity.com/paper/2104.12820/) (2021)
- [Safe Linear Thompson Sampling with Side Information](https://mlsanity.com/paper/1911.02156/) (2019)
- [Optimizing for the Future in Non-Stationary MDPs](https://mlsanity.com/paper/2005.08158/) (2020)
- [On Multi-Armed Bandit Designs for Dose-Finding Clinical Trials](https://mlsanity.com/paper/1903.07082/) (2019)
- [Off-Policy Risk Assessment in Contextual Bandits](https://mlsanity.com/paper/2104.08977/) (2021)
- [Safe Linear Stochastic Bandits](https://mlsanity.com/paper/1911.09501/) (2019)
- [Exploration vs Exploitation vs Safety: Risk-averse Multi-Armed Bandits](https://mlsanity.com/paper/1401.1123/) (2014)
- [Deep Reinforcement Learning for Closed-Loop Blood Glucose Control](https://mlsanity.com/paper/2009.09051/) (2020)
- [Risk-Averse Multi-Armed Bandit Problems under Mean-Variance Measure](https://mlsanity.com/paper/1604.05257/) (2016)
- [Basal Glucose Control in Type 1 Diabetes using Deep Reinforcement Learning: An In Silico Validation](https://mlsanity.com/paper/2005.09059/) (2020)
- [Interpretable Dynamic Treatment Regimes](https://mlsanity.com/paper/1606.01472/) (2016)
- [Risk-Averse Stochastic Convex Bandit](https://mlsanity.com/paper/1810.00737/) (2018)
- [Learning for Dose Allocation in Adaptive Clinical Trials with Safety Constraints](https://mlsanity.com/paper/2006.05026/) (2020)
- [Adaptive Rate of Convergence of Thompson Sampling for Gaussian Process Optimization](https://mlsanity.com/paper/1705.06808/) (2017)
- [Contextual Constrained Learning for Dose-Finding Clinical Trials](https://mlsanity.com/paper/2001.02463/) (2020)
- [Target Tracking for Contextual Bandits: Application to Demand Side Management](https://mlsanity.com/paper/1901.09532/) (2019)
- [SDF-Bayes: Cautious Optimism in Safe Dose-Finding Clinical Trials with Drug Combinations and Heterogeneous Patient Groups](https://mlsanity.com/paper/2101.10998/) (2021)

## Built on by (most cited first, 5 of 5)

- [Transfer Learning for Contextual Multi-armed Bandits](https://mlsanity.com/paper/2211.12612/) (2022)
- [Pure Exploration in Bandits with Linear Constraints](https://mlsanity.com/paper/2306.12774/) (2023)
- [Robust Bayesian Satisficing](https://mlsanity.com/paper/2308.08291/) (2023)
- [Conformal Risk-Averse Decision Making with Action Conditional Guarantee](https://mlsanity.com/paper/2606.05551/) (2026)
- [Some performance considerations when using multi-armed bandit algorithms in the presence of missing data](https://mlsanity.com/paper/2205.03820/) (2022)

## Abstract (arXiv)

Finding an optimal individualized treatment regimen is considered one of the most challenging precision medicine problems. Various patient characteristics influence the response to the treatment, and hence, there is no one-size-fits-all regimen. Moreover, the administration of an unsafe dose during the treatment can have adverse effects on health. Therefore, a treatment model must ensure patient safety while efficiently optimizing the course of therapy. We study a prevalent medical problem where the treatment aims to keep a physiological variable in a safe range and preferably close to a target level, which we refer to as leveling. Such a task may be relevant in numerous other domains as well. We propose ESCADA, a novel and generic multi-armed bandit (MAB) algorithm tailored for the leveling task, to make safe, personalized, and context-aware dose recommendations. We derive high probability upper bounds on its cumulative regret and safety guarantees. Following ESCADA's design, we also describe its Thompson sampling-based counterpart. We discuss why the straightforward adaptations of the classical MAB algorithms such as GP-UCB may not be a good fit for the leveling task. Finally, we make in silico experiments on the bolus-insulin dose allocation problem in type-1 diabetes mellitus disease and compare our algorithms against the famous GP-UCB algorithm, the rule-based dose calculators, and a clinician.