# Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning

arXiv: https://arxiv.org/abs/1901.04670 · HTML: https://mlsanity.com/paper/1901.04670/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2019.

## Built on by (most cited first, 30 of 34)

- [Medical Dead-ends and Learning to Identify High-risk States and Treatments](https://mlsanity.com/paper/2110.04186/) (2021)
- [Semi-Markov Offline Reinforcement Learning for Healthcare](https://mlsanity.com/paper/2203.09365/) (2022)
- [Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applications](https://mlsanity.com/paper/2302.07549/) (2023)
- [Unifying Cardiovascular Modelling with Deep Reinforcement Learning for Uncertainty Aware Control of Sepsis Treatment](https://mlsanity.com/paper/2101.08477/) (2021)
- [Towards Safe Mechanical Ventilation Treatment Using Deep Offline Reinforcement Learning](https://mlsanity.com/paper/2210.02552/) (2022)
- [Offline reinforcement learning with uncertainty for treatment strategies in sepsis](https://mlsanity.com/paper/2107.04491/) (2021)
- [medDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Support](https://mlsanity.com/paper/2505.19785/) (2025)
- [Challenges for Reinforcement Learning in Healthcare](https://mlsanity.com/paper/2103.05612/) (2021)
- [Reinforced Sequential Decision-Making for Sepsis Treatment: The POSNEGDM Framework with Mortality Classifier and Transformer](https://mlsanity.com/paper/2403.07309/) (2024)
- [Deconfounding Actor-Critic Network with Policy Adaptation for Dynamic Treatment Regimes](https://mlsanity.com/paper/2205.09852/) (2022)
- [Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination](https://mlsanity.com/paper/2405.18556/) (2024)
- [Offline Inverse Constrained Reinforcement Learning for Safe-Critical Decision Making in Healthcare](https://mlsanity.com/paper/2410.07525/) (2024)
- [Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey](https://mlsanity.com/paper/2108.04087/) (2021)
- [Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimes](https://mlsanity.com/paper/2407.00364/) (2024)
- [Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI](https://mlsanity.com/paper/2508.21101/) (2025)
- [Formally Verifying and Explaining Sepsis Treatment Policies with COOL-MC](https://mlsanity.com/paper/2602.14505/) (2026)
- [Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care](https://mlsanity.com/paper/2306.08044/) (2023)
- [How Consistent are Clinicians? Evaluating the Predictability of Sepsis Disease Progression with Dynamics Models](https://mlsanity.com/paper/2404.07148/) (2024)
- [Empowering Clinicians with Medical Decision Transformers: A Framework for Sepsis Treatment](https://mlsanity.com/paper/2407.19380/) (2024)
- [Off-Policy Evaluation Under Nonignorable Missing Data](https://mlsanity.com/paper/2507.06961/) (2025)
- [The hidden risks of temporal resampling in clinical reinforcement learning](https://mlsanity.com/paper/2602.06603/) (2026)
- [EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins](https://mlsanity.com/paper/2607.08793/) (2026)
- [Case-based off-policy policy evaluation using prototype learning](https://mlsanity.com/paper/2111.11113/) (2021)
- [Identifying Differential Patient Care Through Inverse Intent Inference](https://mlsanity.com/paper/2411.07372/) (2024)
- [Reinforcement Learning For Survival, A Clinically Motivated Method For Critically Ill Patients](https://mlsanity.com/paper/2207.08040/) (2022)
- [Deep Attention Q-Network for Personalized Treatment Recommendation](https://mlsanity.com/paper/2307.01519/) (2023)
- [Reinforcement Logic Rule Learning for Temporal Point Processes](https://mlsanity.com/paper/2308.06094/) (2023)
- [Nonparametric Additive Value Functions: Interpretable Reinforcement Learning with an Application to Surgical Recovery](https://mlsanity.com/paper/2308.13135/) (2023)
- [Reinforcement learning in large, structured action spaces: A simulation study of decision support for spinal cord injury rehabilitation](https://mlsanity.com/paper/2310.14976/) (2023)
- [Learning optimal treatment strategies for intraoperative hypotension using deep reinforcement learning](https://mlsanity.com/paper/2505.21596/) (2025)

## Abstract (arXiv)

Sepsis is the leading cause of mortality in the ICU. It is challenging to manage because individual patients respond differently to treatment. Thus, tailoring treatment to the individual patient is essential for the best outcomes. In this paper, we take steps toward this goal by applying a mixture-of-experts framework to personalize sepsis treatment. The mixture model selectively alternates between neighbor-based (kernel) and deep reinforcement learning (DRL) experts depending on patient's current history. On a large retrospective cohort, this mixture-based approach outperforms physician, kernel only, and DRL-only experts.