# Model Based Reinforcement Learning for Personalized Heparin Dosing

arXiv: https://arxiv.org/abs/2304.10000 · HTML: https://mlsanity.com/paper/2304.10000/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 90 references that are in the map.
- First posted on arXiv 19 Apr 2023.

## Builds on (most cited first, 12 of 12)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Reinforcement Learning in Healthcare: A Survey](https://mlsanity.com/paper/1908.08796/) (2019)
- [Safe Exploration of State and Action Spaces in Reinforcement Learning](https://mlsanity.com/paper/1402.0560/) (2014)
- [Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data](https://mlsanity.com/paper/1801.09271/) (2018)
- [Behavioral Analytics for Myopic Agents](https://mlsanity.com/paper/1702.05496/) (2017)
- [Reliable Off-policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/2011.04102/) (2020)
- [An Approximate Dynamic Programming Algorithm for Monotone Value Functions](https://mlsanity.com/paper/1401.1590/) (2014)
- [Regret Analysis of Learning-Based MPC with Partially-Unknown Cost Function](https://mlsanity.com/paper/2108.02307/) (2021)
- [Personalized Treatment for Coronary Artery Disease Patients: A Machine Learning Approach](https://mlsanity.com/paper/1910.08483/) (2019)
- [Reinforcement learning and Bayesian data assimilation for model-informed precision dosing in oncology](https://mlsanity.com/paper/2006.01061/) (2020)

## Built on by (most cited first, 5 of 5)

- [Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI](https://mlsanity.com/paper/2508.21101/) (2025)
- [Power Constrained Nonstationary Bandits with Habituation and Recovery Dynamics](https://mlsanity.com/paper/2511.02944/) (2025)
- [An Adaptive Optimization Approach to Personalized Financial Incentives in Mobile Behavioral Weight Loss Interventions](https://mlsanity.com/paper/2307.00444/) (2023)
- [Applications of 0-1 Neural Networks in Prescription and Prediction](https://mlsanity.com/paper/2402.18851/) (2024)
- [Optimizing Digital Therapeutic Interventions: Online Learning under Endogenous Adherence](https://mlsanity.com/paper/2605.24261/) (2026)

## Abstract (arXiv)

A key challenge in sequential decision making is optimizing systems safely under partial information. While much of the literature has focused on the cases of either partially known states or partially known dynamics, it is further exacerbated in cases where both states and dynamics are partially known. Computing heparin doses for patients fits this paradigm since the concentration of heparin in the patient cannot be measured directly and the rates at which patients metabolize heparin vary greatly between individuals. While many proposed solutions are model free, they require complex models and have difficulty ensuring safety. However, if some of the structure of the dynamics is known, a model based approach can be leveraged to provide safe policies. In this paper we propose such a framework to address the challenge of optimizing personalized heparin doses. We use a predictive model parameterized individually by patient to predict future therapeutic effects. We then leverage this model using a scenario generation based approach that is capable of ensuring patient safety. We validate our models with numerical experiments by comparing the predictive capabilities of our model against existing machine learning techniques and demonstrating how our dosing algorithm can treat patients in a simulated ICU environment.