mlsanityData to 7 Oct 2026

Offline reinforcement learning with uncertainty for treatment strategies in sepsis

arXiv 2107.04491 ↗First posted 9 Jul 2021cs.LG cs.AIcited by 9

On the map: Offline RL and imitation learning from data → Safe offline RL

Abstract (arXiv)

Guideline-based treatment for sepsis and septic shock is difficult because sepsis is a disparate range of life-threatening organ dysfunctions whose pathophysiology is not fully understood. Early intervention in sepsis is crucial for patient outcome, yet those interventions have adverse effects and are frequently overadministered. Greater personalization is necessary, as no single action is suitable for all patients. We present a novel application of reinforcement learning in which we identify optimal recommendations for sepsis treatment from data, estimate their confidence level, and identify treatment options infrequently observed in training data. Rather than a single recommendation, our method can present several treatment options. We examine learned policies and discover that reinforcement learning is biased against aggressive intervention due to the confounding relationship between mortality and level of treatment received. We mitigate this bias using subspace learning, and develop methodology that can yield more accurate learning policies across healthcare applications.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 9 of its 46 references that are in the map. First posted on arXiv 9 Jul 2021.

Builds on

9 of 46 references.

References · 9

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  4. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  5. 2016Deep Exploration via Bootstrapped DQNcited by 518
  6. 2017Uncertainty-Aware Reinforcement Learning for Collision Avoidancecited by 86
  7. 2019Explainable artificial intelligence model to predict acute critical illness from electronic health recordscited by 42
  8. 2019Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learningcited by 34
  9. 2019Optimizing Sequential Medical Treatments with Auto-Encoding Heuristic Search in POMDPscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 2; 2024: 1; 2025: 0; 2026: 3.

9 papers.

  1. 2023Elastic Decision Transformercited by 66
  2. 2022Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetescited by 22
  3. 2022The Health Gym: Synthetic Health-Related Datasets for the Development of Reinforcement Learning Algorithmscited by 19
  4. 2022Generating Synthetic Clinical Data that Capture Class Imbalanced Distributions with Generative Adversarial Networks: Example using Antiretroviral Therapy for HIVcited by 12
  5. 2023Synthetic Health-related Longitudinal Data with Mixed-type Variables Generated using Diffusion Modelscited by 9
  6. 2026Formally Verifying and Explaining Sepsis Treatment Policies with COOL-MCcited by 3
  7. 2024How Consistent are Clinicians? Evaluating the Predictability of Sepsis Disease Progression with Dynamics Modelscited by 2
  8. 2026The hidden risks of temporal resampling in clinical reinforcement learningcited by 1
  9. 2026EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twinscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2022Learning Optimal Treatment Strategies for Sepsis Using Offline Reinforcement Learning in Continuous Space
  2. 2024Empowering Clinicians with Medical Decision Transformers: A Framework for Sepsis Treatment
  3. 2025Exploring Time-Step Size in Reinforcement Learning for Sepsis Treatment
  4. 2024Identifying Differential Patient Care Through Inverse Intent Inference
  5. 2022A Conservative Q-Learning approach for handling distribution shift in sepsis treatment strategies
  6. 2026Attention-Based Offline Reinforcement Learning and Clustering for Interpretable Sepsis Treatment
  7. 2026Offline Reinforcement Learning for Hemodynamic Management of Sepsis in the ICU: a MIMIC-IV Study with Dual Off-Policy Evaluation
  8. 2024Reinforced Sequential Decision-Making for Sepsis Treatment: The POSNEGDM Framework with Mortality Classifier and Transformer