# Challenges for Reinforcement Learning in Healthcare

arXiv: https://arxiv.org/abs/2103.05612 · HTML: https://mlsanity.com/paper/2103.05612/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 54 references that are in the map.
- First posted on arXiv 9 Mar 2021.

## Builds on (most cited first, 16 of 16)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Doubly Robust Off-policy Value Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1511.03722/) (2015)
- [Learning to Explain: An Information-Theoretic Perspective on Model Interpretation](https://mlsanity.com/paper/1802.07814/) (2018)
- [More Robust Doubly Robust Off-policy Evaluation](https://mlsanity.com/paper/1802.03493/) (2018)
- [Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models](https://mlsanity.com/paper/1905.05824/) (2019)
- [Deep Reinforcement Learning for Sepsis Treatment](https://mlsanity.com/paper/1711.09602/) (2017)
- [Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1705.08422/) (2017)
- [Evaluating Reinforcement Learning Algorithms in Observational Health Settings](https://mlsanity.com/paper/1805.12298/) (2018)
- [A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units](https://mlsanity.com/paper/1704.06300/) (2017)
- [Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning](https://mlsanity.com/paper/1901.04670/) (2019)
- [Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record](https://mlsanity.com/paper/1810.04793/) (2018)
- [Model-Based Reinforcement Learning for Sepsis Treatment](https://mlsanity.com/paper/1811.09602/) (2018)
- [Representation and Reinforcement Learning for Personalized Glycemic Control in Septic Patients](https://mlsanity.com/paper/1712.00654/) (2017)
- [An Optimal Policy for Patient Laboratory Tests in Intensive Care Units](https://mlsanity.com/paper/1808.04679/) (2018)
- [Introduction to the "Industrial Benchmark"](https://mlsanity.com/paper/1610.03793/) (2016)

## Built on by (most cited first, 7 of 7)

- [Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunities](https://mlsanity.com/paper/2210.11846/) (2022)
- [Uncertainty-driven Trajectory Truncation for Data Augmentation in Offline Reinforcement Learning](https://mlsanity.com/paper/2304.04660/) (2023)
- [Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI](https://mlsanity.com/paper/2508.21101/) (2025)
- [OCMDP: Observation-Constrained Markov Decision Process](https://mlsanity.com/paper/2411.07087/) (2024)
- [Policy Resilience to Environment Poisoning Attacks on Reinforcement Learning](https://mlsanity.com/paper/2304.12151/) (2023)
- [Safe and Interpretable Estimation of Optimal Treatment Regimes](https://mlsanity.com/paper/2310.15333/) (2023)
- [Inverse Reinforcement Learning with Multiple Planning Horizons](https://mlsanity.com/paper/2409.18051/) (2024)

## Abstract (arXiv)

Many healthcare decisions involve navigating through a multitude of treatment options in a sequential and iterative manner to find an optimal treatment pathway with the goal of an optimal patient outcome. Such optimization problems may be amenable to reinforcement learning. A reinforcement learning agent could be trained to provide treatment recommendations for physicians, acting as a decision support tool. However, a number of difficulties arise when using RL beyond benchmark environments, such as specifying the reward function, choosing an appropriate state representation and evaluating the learned policy.