mlsanityData to 7 Oct 2026

Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach

arXiv 1705.08422 ↗First posted 23 May 2017cs.LGcited by 68

Not on the map. Why?

Abstract (arXiv)

Sepsis is a leading cause of mortality in intensive care units (ICUs) and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. Understanding more about a patient's physiological state at a given time could hold the key to effective treatment policies. In this work, we propose a new approach to deduce optimal treatment policies for septic patients by using continuous state-space models and deep reinforcement learning. Learning treatment policies over continuous spaces is important, because we retain more of the patient's physiological information. Our model is able to learn clinically interpretable treatment policies, similar in important aspects to the treatment policies of physicians. Evaluating our algorithm on past ICU patient data, we find that our model could reduce patient mortality in the hospital by up to 3.6% over observed clinical policies, from a baseline mortality of 13.7%. The learned treatment policies could be used to aid intensive care clinicians in medical decision making and improve the likelihood of patient survival.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 12; 2022: 9; 2023: 12; 2024: 11; 2025: 8; 2026: 15.

68 papers.

Sort:

By area:

  1. 2021Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settingscited by 68
  2. 2021Instrumental Variable Value Iteration for Causal Offline Reinforcement Learningcited by 35
  3. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
  4. 2021Medical Dead-ends and Learning to Identify High-risk States and Treatmentscited by 28
  5. 2022Offline Reinforcement Learning with Differential Privacycited by 19
  6. 2022Semi-Markov Offline Reinforcement Learning for Healthcarecited by 18
  7. 2023Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applicationscited by 17
  8. 2022Near-Optimal Differentially Private Reinforcement Learningcited by 16
  9. 2020Expert-Supervised Reinforcement Learning for Offline Policy Learning and Evaluationcited by 14
  10. 2022Conditional Generation of Medical Time Series for Extrapolation to Underrepresented Populationscited by 14
  11. 2022Federated Offline Reinforcement Learningcited by 13
  12. 2021Multi-source Learning via Completion of Block-wise Overlapping Noisy Matricescited by 10
38 more
  1. 2021Challenges for Reinforcement Learning in Healthcarecited by 7
  2. 2021Feature-Based Interpretable Reinforcement Learning based on State-Transition Modelscited by 6
  3. 2021Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learningcited by 6
  4. 2022Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventionscited by 6
  5. 2023Risk Sensitive Dead-end Identification in Safety-Critical Offline Reinforcement Learningcited by 6
  6. 2024Reinforced Sequential Decision-Making for Sepsis Treatment: The POSNEGDM Framework with Mortality Classifier and Transformercited by 6
  7. 2022Optimal Treatment Regimes for Proximal Causal Learningcited by 5
  8. 2021Learning without Knowing: Unobserved Context in Continuous Transfer Reinforcement Learningcited by 4
  9. 2024Finite-Time Convergence and Sample Complexity of Actor-Critic Multi-Objective Reinforcement Learningcited by 4
  10. 2024Offline Inverse Constrained Reinforcement Learning for Safe-Critical Decision Making in Healthcarecited by 4
  11. 2025Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategiescited by 4
  12. 2021Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Surveycited by 3
  13. 2021Optimal discharge of patients from intensive care via a data-driven policy learning frameworkcited by 3
  14. 2024Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimescited by 3
  15. 2025Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AIcited by 3
  16. 2026Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policiescited by 3
  17. 2022Adaptive Semi-Supervised Inference for Optimal Treatment Decisions with Electronic Medical Record Datacited by 2
  18. 2022Relative Sparsity for Medical Decision Problemscited by 2
  19. 2023Towards Real-World Applications of Personalized Anesthesia Using Policy Constraint Q Learning for Propofol Infusion Controlcited by 2
  20. 2023Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Carecited by 2
  21. 2024Measurement Scheduling for ICU Patients with Offline Reinforcement Learningcited by 2
  22. 2024How Consistent are Clinicians? Evaluating the Predictability of Sepsis Disease Progression with Dynamics Modelscited by 2
  23. 2024Deep State-Space Generative Model For Correlated Time-to-Event Predictionscited by 2
  24. 2025Off-Policy Evaluation Under Nonignorable Missing Datacited by 2
  25. 2025Guardian-regularized Safe Offline Reinforcement Learning for Smart Weaning of Mechanical Circulatory Devicescited by 2
  26. 2026A Generalized Apprenticeship Learning Framework for Capturing Evolving Student Pedagogical Strategiescited by 2
  27. 2021Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learningcited by 1
  28. 2023A optimization framework for herbal prescription planning based on deep reinforcement learningcited by 1
  29. 2023An Offline Time-aware Apprenticeship Learning Framework for Evolving Reward Functionscited by 1
  30. 2024Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learningcited by 1
  31. 2026The hidden risks of temporal resampling in clinical reinforcement learningcited by 1
  32. 2026CSRA: Controlled Spectral Residual Augmentation for Robust Sepsis Predictioncited by 1
  33. 2026EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twinscited by 1
  34. 2021An Analysis of Reinforcement Learning for Malaria Controlcited by 0
  35. 2023Kernel-Based Distributed Q-Learning: A Scalable Reinforcement Learning Approach for Dynamic Treatment Regimescited by 0
  36. 2023Offline Policy Evaluation for Reinforcement Learning with Adaptively Collected Datacited by 0
  37. 2023Deep Attention Q-Network for Personalized Treatment Recommendationcited by 0
  38. 2023Nonparametric Additive Value Functions: Interpretable Reinforcement Learning with an Application to Surgical Recoverycited by 0

50 most cited of 68 All 68 (JSON)