# Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimes

arXiv: https://arxiv.org/abs/2407.00364 · HTML: https://mlsanity.com/paper/2407.00364/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for clinical treatment and surgery, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 115 references that are in the map.
- First posted on arXiv 29 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL for clinical treatment and surgery](https://mlsanity.com/deeprl/rl-for-clinical-treatment-and-surgery/)

## Builds on (most cited first, 22 of 22)

- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Reinforcement Learning in Healthcare: A Survey](https://mlsanity.com/paper/1908.08796/) (2019)
- [Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation](https://mlsanity.com/paper/1807.01473/) (2018)
- [A Review of Off-Policy Evaluation in Reinforcement Learning](https://mlsanity.com/paper/2212.06355/) (2022)
- [Deep Reinforcement Learning for Sepsis Treatment](https://mlsanity.com/paper/1711.09602/) (2017)
- [Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1705.08422/) (2017)
- [Estimating Dynamic Treatment Regimes in Mobile Health Using V-learning](https://mlsanity.com/paper/1611.03531/) (2016)
- [A Survey on Interactive Reinforcement Learning: Design Principles and Open Challenges](https://mlsanity.com/paper/2105.12949/) (2021)
- [Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning](https://mlsanity.com/paper/1901.04670/) (2019)
- [Clinician-in-the-Loop Decision Making: Reinforcement Learning with Near-Optimal Set-Valued Policies](https://mlsanity.com/paper/2007.12678/) (2020)
- [Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data](https://mlsanity.com/paper/1801.09271/) (2018)
- [Model-Based Reinforcement Learning for Sepsis Treatment](https://mlsanity.com/paper/1811.09602/) (2018)
- [Expert-Supervised Reinforcement Learning for Offline Policy Learning and Evaluation](https://mlsanity.com/paper/2006.13189/) (2020)
- [Optimizing Sequential Medical Treatments with Auto-Encoding Heuristic Search in POMDPs](https://mlsanity.com/paper/1905.07465/) (2019)
- [Optimal regimes for algorithm-assisted human decision-making](https://mlsanity.com/paper/2203.03020/) (2022)
- [Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation](https://mlsanity.com/paper/2003.06474/) (2020)
- [Non-Deterministic Policies in Markovian Decision Processes](https://mlsanity.com/paper/1401.3871/) (2014)
- [The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making](https://mlsanity.com/paper/1805.11548/) (2018)
- [Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventions](https://mlsanity.com/paper/2203.02605/) (2022)
- [Estimation and Optimization of Composite Outcomes](https://mlsanity.com/paper/1711.10581/) (2017)
- [A Conservative Q-Learning approach for handling distribution shift in sepsis treatment strategies](https://mlsanity.com/paper/2203.13884/) (2022)
- [Robust Hybrid Learning for Estimating Personalized Dynamic Treatment Regimens](https://mlsanity.com/paper/1611.02314/) (2016)

## Built on by (most cited first, 3 of 3)

- [Towards Optimal Differentially Private Regret Bounds in Linear MDPs](https://mlsanity.com/paper/2504.09339/) (2025)
- [Treatment, evidence, imitation, and chat](https://mlsanity.com/paper/2506.23040/) (2025)
- [Near-Equivalent Q-learning Policies for Dynamic Treatment Regimes](https://mlsanity.com/paper/2603.19440/) (2026)

## Abstract (arXiv)

The goal of precision medicine is to provide individualized treatment at each stage of chronic diseases, a concept formalized by Dynamic Treatment Regimes (DTR). These regimes adapt treatment strategies based on decision rules learned from clinical data to enhance therapeutic effectiveness. Reinforcement Learning (RL) algorithms allow to determine these decision rules conditioned by individual patient data and their medical history. The integration of medical expertise into these models makes possible to increase confidence in treatment recommendations and facilitate the adoption of this approach by healthcare professionals and patients. In this work, we examine the mathematical foundations of RL, contextualize its application in the field of DTR, and present an overview of methods to improve its effectiveness by integrating medical expertise.