# Optimizing Warfarin Dosing Using Contextual Bandit: An Offline Policy Learning and Evaluation Method

arXiv: https://arxiv.org/abs/2402.11123 · HTML: https://mlsanity.com/paper/2402.11123/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 8 of 8)

- [Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism](https://mlsanity.com/paper/2103.12021/) (2021)
- [Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1911.06854/) (2019)
- [Offline A/B testing for Recommender Systems](https://mlsanity.com/paper/1801.07030/) (2018)
- [Deep Reinforcement Learning for Sepsis Treatment](https://mlsanity.com/paper/1711.09602/) (2017)
- [Evaluating Reinforcement Learning Algorithms in Observational Health Settings](https://mlsanity.com/paper/1805.12298/) (2018)
- [Adapting multi-armed bandits policies to contextual bandits scenarios](https://mlsanity.com/paper/1811.04383/) (2018)
- [Online Learning to Estimate Warfarin Dose with Contextual Linear Bandits](https://mlsanity.com/paper/1907.05496/) (2019)
- [Estimation of Warfarin Dosage with Reinforcement Learning](https://mlsanity.com/paper/2109.07564/) (2021)

## Abstract (arXiv)

Warfarin, an anticoagulant medication, is formulated to prevent and address conditions associated with abnormal blood clotting, making it one of the most prescribed drugs globally. However, determining the suitable dosage remains challenging due to individual response variations, and prescribing an incorrect dosage may lead to severe consequences. Contextual bandit and reinforcement learning have shown promise in addressing this issue. Given the wide availability of observational data and safety concerns of decision-making in healthcare, we focused on using exclusively observational data from historical policies as demonstrations to derive new policies; we utilized offline policy learning and evaluation in a contextual bandit setting to establish the optimal personalized dosage strategy. Our learned policies surpassed these baseline approaches without genotype inputs, even when given a suboptimal demonstration, showcasing promising application potential.