# A Contextual-bandit-based Approach for Informed Decision-making in Clinical Trials

arXiv: https://arxiv.org/abs/1809.00258 · HTML: https://mlsanity.com/paper/1809.00258/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2018.

## Built on by (most cited first, 18 of 18)

- [Nash Regret Guarantees for Linear Bandits](https://mlsanity.com/paper/2310.02023/) (2023)
- [Multi-disciplinary fairness considerations in machine learning for clinical trials](https://mlsanity.com/paper/2205.08875/) (2022)
- [Doubly Adversarial Federated Bandits](https://mlsanity.com/paper/2301.09223/) (2023)
- [SDF-Bayes: Cautious Optimism in Safe Dose-Finding Clinical Trials with Drug Combinations and Heterogeneous Patient Groups](https://mlsanity.com/paper/2101.10998/) (2021)
- [Towards Regulatory-Confirmed Adaptive Clinical Trials: Machine Learning Opportunities and Solutions](https://mlsanity.com/paper/2503.09226/) (2025)
- [Worst-case Performance of Greedy Policies in Bandits with Imperfect Context Observations](https://mlsanity.com/paper/2204.04773/) (2022)
- [Clustering Context in Off-Policy Evaluation](https://mlsanity.com/paper/2502.21304/) (2025)
- [Provable Anytime Ensemble Sampling Algorithms in Nonlinear Contextual Bandits](https://mlsanity.com/paper/2510.10730/) (2025)
- [Tailored Behavior-Change Messaging for Physical Activity: Integrating Contextual Bandits and Large Language Models](https://mlsanity.com/paper/2506.07275/) (2025)
- [Prior-informed optimization of treatment recommendation via bandit algorithms trained on large language model-processed historical records](https://mlsanity.com/paper/2510.19014/) (2025)
- [Quantum-Enhanced Neural Contextual Bandit Algorithms](https://mlsanity.com/paper/2601.02870/) (2026)
- [Shuffle and Joint Differential Privacy for Generalized Linear Contextual Bandits](https://mlsanity.com/paper/2602.00417/) (2026)
- [Active Learning for Stochastic Contextual Linear Bandits](https://mlsanity.com/paper/2605.24803/) (2026)
- [AI in Pharma for Personalized Sequential Decision-Making: Methods, Applications and Opportunities](https://mlsanity.com/paper/2311.18725/) (2023)
- [Multi-armed Bandits with Missing Outcome](https://mlsanity.com/paper/2411.05661/) (2024)
- [DAL: A Practical Prior-Free Black-Box Framework for Piecewise Stationary Bandits](https://mlsanity.com/paper/2501.19401/) (2025)
- [Inverse Contextual Bandits without Rewards: Learning from a Non-Stationary Learner via Suffix Imitation](https://mlsanity.com/paper/2603.03778/) (2026)
- [Mixing Makes Markovian Contexts Cheap for Linear Bandits](https://mlsanity.com/paper/2603.12530/) (2026)

## Abstract (arXiv)

Clinical trials involving multiple treatments utilize randomization of the treatment assignments to enable the evaluation of treatment efficacies in an unbiased manner. Such evaluation is performed in post hoc studies that usually use supervised-learning methods that rely on large amounts of data collected in a randomized fashion. That approach often proves to be suboptimal in that some participants may suffer and even die as a result of having not received the most appropriate treatments during the trial. Reinforcement-learning methods improve the situation by making it possible to learn the treatment efficacies dynamically during the course of the trial, and to adapt treatment assignments accordingly. Recent efforts using textitmulti-arm bandits, a type of reinforcement-learning methods, have focused on maximizing clinical outcomes for a population that was assumed to be homogeneous. However, those approaches have failed to account for the variability among participants that is becoming increasingly evident as a result of recent clinical-trial-based studies. We present a contextual-bandit-based online treatment optimization algorithm that, in choosing treatments for new participants in the study, takes into account not only the maximization of the clinical outcomes but also the patient characteristics. We evaluated our algorithm using a real clinical trial dataset from the International Stroke Trial. The results of our retrospective analysis indicate that the proposed approach performs significantly better than either a random assignment of treatments (the current gold standard) or a multi-arm-bandit-based approach, providing substantial gains in the percentage of participants who are assigned the most suitable treatments. The contextual-bandit and multi-arm bandit approaches provide 72.63% and 64.34% gains, respectively, compared to a random assignment.