mlsanityData to 7 Oct 2026

Exploring a Graph-based Approach to Offline Reinforcement Learning for Sepsis Treatment

arXiv 2509.03393 ↗First posted 3 Sep 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Sepsis is a serious, life-threatening condition. When treating sepsis, it is challenging to determine the correct amount of intravenous fluids and vasopressors for a given patient. While automated reinforcement learning (RL)-based methods have been used to support these decisions with promising results, previous studies have relied on relational data. Given the complexity of modern healthcare data, representing data as a graph may provide a more natural and effective approach. This study models patient data from the well-known MIMIC-III dataset as a heterogeneous graph that evolves over time. Subsequently, we explore two Graph Neural Network architectures - GraphSAGE and GATv2 - for learning patient state representations, adopting the approach of decoupling representation learning from policy learning. The encoders are trained to produce latent state representations, jointly with decoders that predict the next patient state. These representations are then used for policy learning with the dBCQ algorithm. The results of our experimental evaluation confirm the potential of a graph-based approach, while highlighting the complexity of representation learning in this domain.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 37 references that are in the map. First posted on arXiv 3 Sep 2025.

Builds on

13 of 37 references.

References · 13

  1. 2017Graph Attention Networkscited by 8,854
  2. 2017Inductive Representation Learning on Large Graphscited by 6,284
  3. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  4. 2021How Attentive are Graph Attention Networks?cited by 818
  5. 2020A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sourcescited by 126
  6. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 108
  7. 2019Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use casecited by 53
  8. 2020An Empirical Study of Representation Learning for Reinforcement Learning in Healthcarecited by 34
  9. 2022Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applicationscited by 27
  10. 2021Predicting Patient Outcomes with Graph Representation Learningcited by 21
  11. 2019Optimizing Sequential Medical Treatments with Auto-Encoding Heuristic Search in POMDPscited by 12
  12. 2024Reinforced Sequential Decision-Making for Sepsis Treatment: The POSNEGDM Framework with Mortality Classifier and Transformercited by 6
1 more
  1. 2020Deep Learning with Heterogeneous Graph Embeddings for Mortality Prediction from Electronic Health Recordscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Attention-Based Offline Reinforcement Learning and Clustering for Interpretable Sepsis Treatment
  2. 2021Offline reinforcement learning with uncertainty for treatment strategies in sepsis
  3. 2022Learning Optimal Treatment Strategies for Sepsis Using Offline Reinforcement Learning in Continuous Space
  4. 2025Exploring Time-Step Size in Reinforcement Learning for Sepsis Treatment
  5. 2025MORE-CLEAR: Multimodal Offline Reinforcement learning for Clinical notes Leveraged Enhanced State Representation
  6. 2026Offline Reinforcement Learning for Hemodynamic Management of Sepsis in the ICU: a MIMIC-IV Study with Dual Off-Policy Evaluation
  7. 2019Sepsis World Model: A MIMIC-based OpenAI Gym "World Model" Simulator for Sepsis Treatment
  8. 2025Stable CDE Autoencoders with Acuity Regularization for Offline Reinforcement Learning in Sepsis Treatment