mlsanityData to 14 Oct 2026

An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare

arXiv 2011.11235 ↗First posted 23 Nov 2020cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning (RL) has recently been applied to sequential estimation and prediction problems identifying and developing hypothetical treatment strategies for septic patients, with a particular focus on offline learning with observational data. In practice, successful RL relies on informative latent states derived from sequential observations to develop optimal treatment strategies. To date, how best to construct such states in a healthcare setting is an open question. In this paper, we perform an empirical study of several information encoding architectures using data from septic patients in the MIMIC-III dataset to form representations of a patient state. We evaluate the impact of representation dimension, correlations with established acuity scores, and the treatment policies derived from them. We find that sequentially formed state representations facilitate effective policy learning in batch settings, validating a more thoughtful approach to representation learning that remains faithful to the sequential and partial nature of healthcare data.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 2; 2023: 9; 2024: 7; 2025: 8; 2026: 3.

34 papers.

Sort:

By area:

  1. 2021Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settingscited by 68
  2. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  3. 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 42
  4. 2021Neighborhood Contrastive Learning Applied to Online Patient Monitoringcited by 32
  5. 2021Medical Dead-ends and Learning to Identify High-risk States and Treatmentscited by 28
  6. 2023Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applicationscited by 17
  7. 2021Unifying Cardiovascular Modelling with Deep Reinforcement Learning for Uncertainty Aware Control of Sepsis Treatmentcited by 16
  8. 2024Offline Multitask Representation Learning for Reinforcement Learningcited by 14
  9. 2023Robust Fitted-Q-Evaluation and Iteration under Sequentially Exogenous Unobserved Confounderscited by 13
  10. 2023Finding Counterfactually Optimal Action Sequences in Continuous State Spacescited by 13
  11. 2023Temporal Supervised Contrastive Learning for Modeling Patient Risk Progressioncited by 10
  12. 2025medDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Supportcited by 9
22 more
  1. 2023Risk Sensitive Dead-end Identification in Safety-Critical Offline Reinforcement Learningcited by 6
  2. 2023Approximate Information States for Worst-Case Control and Learning in Uncertain Systemscited by 3
  3. 2023Enabling A Network AI Gym for Autonomous Cyber Agentscited by 3
  4. 2024Offline Reinforcement Learning with Domain-Unlabeled Datacited by 3
  5. 2025Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AIcited by 3
  6. 2024How Consistent are Clinicians? Evaluating the Predictability of Sepsis Disease Progression with Dynamics Modelscited by 2
  7. 2024Empowering Clinicians with Medical Decision Transformers: A Framework for Sepsis Treatmentcited by 2
  8. 2024OCMDP: Observation-Constrained Markov Decision Processcited by 2
  9. 2025Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learningcited by 2
  10. 2025MORE-CLEAR: Multimodal Offline Reinforcement learning for Clinical notes Leveraged Enhanced State Representationcited by 2
  11. 2025MIMIC-Sepsis: A Curated Benchmark for Modeling and Learning from Sepsis Trajectories in the ICUcited by 2
  12. 2026The hidden risks of temporal resampling in clinical reinforcement learningcited by 2
  13. 2022Deep Normed Embeddings for Patient Representationcited by 1
  14. 2023A Framework for Empowering Reinforcement Learning Agents with Causal Analysis: Enhancing Automated Cryptocurrency Tradingcited by 1
  15. 2024Development and Validation of Heparin Dosing Policies Using an Offline Reinforcement Learning Algorithmcited by 1
  16. 2024Identifying Differential Patient Care Through Inverse Intent Inferencecited by 1
  17. 2022Reinforcement Learning For Survival, A Clinically Motivated Method For Critically Ill Patientscited by 0
  18. 2025Q-function Decomposition with Intervention Semantics with Factored Action Spacescited by 0
  19. 2025Stable CDE Autoencoders with Acuity Regularization for Offline Reinforcement Learning in Sepsis Treatmentcited by 0
  20. 2025Exploring a Graph-based Approach to Offline Reinforcement Learning for Sepsis Treatmentcited by 0
  21. 2026Learning Preference-Based Objectives from Clinical Narratives for Dynamic Sepsis Treatmentcited by 0
  22. 2026The Evidence Ladder for Reinforcement Learning in Healthcare: From Retrospective Policies to Trusted Interventionscited by 0