# Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data

arXiv: https://arxiv.org/abs/1801.09271 · HTML: https://mlsanity.com/paper/1801.09271/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jan 2018.

## Built on by (most cited first, 23 of 23)

- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models](https://mlsanity.com/paper/2502.13187/) (2025)
- [Corruption-Robust Offline Reinforcement Learning with General Function Approximation](https://mlsanity.com/paper/2310.14550/) (2023)
- [Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitation](https://mlsanity.com/paper/2411.09891/) (2024)
- [Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settings](https://mlsanity.com/paper/2105.06029/) (2021)
- [Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventions](https://mlsanity.com/paper/2203.02605/) (2022)
- [Model Based Reinforcement Learning for Personalized Heparin Dosing](https://mlsanity.com/paper/2304.10000/) (2023)
- [Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination](https://mlsanity.com/paper/2405.18556/) (2024)
- [Learning Optimal Dynamic Treatment Regimes Using Causal Tree Methods in Medicine](https://mlsanity.com/paper/2204.07124/) (2022)
- [There is no Accuracy-Interpretability Tradeoff in Reinforcement Learning for Mazes](https://mlsanity.com/paper/2206.04266/) (2022)
- [Sharp Gap-Dependent Variance-Aware Regret Bounds for Tabular MDPs](https://mlsanity.com/paper/2506.06521/) (2025)
- [Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimes](https://mlsanity.com/paper/2407.00364/) (2024)
- [Online Decision Making with Generative Action Sets](https://mlsanity.com/paper/2509.25777/) (2025)
- [The hidden risks of temporal resampling in clinical reinforcement learning](https://mlsanity.com/paper/2602.06603/) (2026)
- [A optimization framework for herbal prescription planning based on deep reinforcement learning](https://mlsanity.com/paper/2304.12828/) (2023)
- [Prompting Large Language Models for Supporting the Differential Diagnosis of Anemia](https://mlsanity.com/paper/2409.15377/) (2024)
- [How to Provably Improve Return Conditioned Supervised Learning?](https://mlsanity.com/paper/2506.08463/) (2025)
- [TECM*: A Data-Driven Assessment to Reinforcement Learning Methods and Application to Heparin Treatment Strategy for Surgical Sepsis](https://mlsanity.com/paper/2512.10973/) (2025)
- [An Analysis of Reinforcement Learning for Malaria Control](https://mlsanity.com/paper/2107.08988/) (2021)
- [Context-Aware Composition of Agent Policies by Markov Decision Process Entity Embeddings and Agent Ensembles](https://mlsanity.com/paper/2308.14521/) (2023)
- [Is Offline Decision Making Possible with Only Few Samples? Reliable Decisions in Data-Starved Bandits via Trust Region Enhancement](https://mlsanity.com/paper/2402.15703/) (2024)
- [Artificial Intelligence-based Decision Support Systems for Precision and Digital Health](https://mlsanity.com/paper/2407.16062/) (2024)
- [Where to Intervene: Action Selection in Deep Reinforcement Learning](https://mlsanity.com/paper/2507.04187/) (2025)

## Abstract (arXiv)

This paper presents the first deep reinforcement learning (DRL) framework to estimate the optimal Dynamic Treatment Regimes from observational medical data. This framework is more flexible and adaptive for high dimensional action and state spaces than existing reinforcement learning methods to model real-life complexity in heterogeneous disease progression and treatment choices, with the goal of providing doctor and patients the data-driven personalized decision recommendations. The proposed DRL framework comprises (i) a supervised learning step to predict the most possible expert actions, and (ii) a deep reinforcement learning step to estimate the long-term value function of Dynamic Treatment Regimes. Both steps depend on deep neural networks. As a key motivational example, we have implemented the proposed framework on a data set from the Center for International Bone Marrow Transplant Research (CIBMTR) registry database, focusing on the sequence of prevention and treatments for acute and chronic graft versus host disease after transplantation. In the experimental results, we have demonstrated promising accuracy in predicting human experts' decisions, as well as the high expected reward function in the DRL-based dynamic treatment regimes.