mlsanityData to 14 Oct 2026

medDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Support

arXiv 2505.19785 ↗First posted 26 May 2025cs.LG cs.AIcited by 9

On the map: Deep reinforcement learning methods → RL for clinical treatment and surgery

Abstract (arXiv)

Timely and personalized treatment decisions are essential across a wide range of healthcare settings where patient responses can vary significantly and evolve over time. Clinical data used to support these treatment decisions are often irregularly sampled, where missing data frequencies may implicitly convey information about the patient's condition. Existing Reinforcement Learning (RL) based clinical decision support systems often ignore the missing patterns and distort them with coarse discretization and simple imputation. They are also predominantly model-free and largely depend on retrospective data, which could lead to insufficient exploration and bias by historical behaviors. To address these limitations, we propose medDreamer, a novel model-based reinforcement learning framework for personalized treatment recommendation. medDreamer contains a world model with an Adaptive Feature Integration module that simulates latent patient states from irregular data and a two-phase policy trained on a hybrid of real and imagined trajectories. This enables learning optimal policies that go beyond the sub-optimality of historical clinical decisions, while remaining close to real clinical data. We evaluate medDreamer on both sepsis and mechanical ventilation treatment tasks using two large-scale Electronic Health Records (EHRs) datasets. Comprehensive evaluations show that medDreamer significantly outperforms model-free and model-based baselines in both clinical outcomes and off-policy metrics.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in RL for clinical treatment and surgery, one of 15 directions in Deep reinforcement learning methods. Builds on 40 of its 78 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

40 of 78 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Empowering Clinicians with Medical Decision Transformers: A Framework for Sepsis Treatment28 Jul
  2. 2024DTR-Bench: An in silico Environment and Benchmark Platform for Reinforcement Learning Based Dynamic Treatment Regime28 May
  3. 2024Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination28 May
  4. 2024Model-Free Reinforcement Learning for Automated Fluid Administration in Critical Care11 Jan

Other references · 36

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2018World Modelscited by 1,513
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  7. 2023Mastering Diverse Domains through World Modelscited by 1,253
  8. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  9. 2020Mastering Atari with Discrete World Modelscited by 1,007
  10. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  11. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  12. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
24 more
  1. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 554
  2. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 506
  3. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 370
  4. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  5. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 231
  6. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  7. 2020Neural Rough Differential Equations for Long Time Seriescited by 143
  8. 2018Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendationcited by 101
  9. 2017Deep Reinforcement Learning for Sepsis Treatmentcited by 71
  10. 2022Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equationscited by 59
  11. 2024Mastering Memory Tasks with World Modelscited by 38
  12. 2019Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learningcited by 34
  13. 2020An Empirical Study of Representation Learning for Reinforcement Learning in Healthcarecited by 34
  14. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  15. 2017Robust and Efficient Transfer Learning with Hidden-Parameter Markov Decision Processescited by 29
  16. 2024Improving Agent Behaviors with RL Fine-tuning for Autonomous Drivingcited by 27
  17. 2022Predicting the impact of treatments over time with uncertainty aware neural differential equationscited by 23
  18. 2018Model-Based Reinforcement Learning for Sepsis Treatmentcited by 21
  19. 2025Learning Transformer-based World Models with Contrastive Predictive Codingcited by 20
  20. 2023Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applicationscited by 17
  21. 2022Towards Safe Mechanical Ventilation Treatment Using Deep Offline Reinforcement Learningcited by 16
  22. 2022Exploiting Transformer-based Multitask Learning for the Detection of Media Bias in News Articlescited by 11
  23. 2020Trajectory Inspection: A Method for Iterative Clinician-Driven Design of Reinforcement Learning Studiescited by 5
  24. 2022Deconfounding Actor-Critic Network with Policy Adaptation for Dynamic Treatment Regimescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 8.

9 papers.

  1. 2026Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policiescited by 4
  2. 2026World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applicationscited by 3
  3. 2026Sonata: A Hybrid World Model for Inertial Kinematics under Clinical Data Scarcitycited by 2
  4. 2026Agentifying Patient Dynamics within LLMs through Interacting with Clinical World Modelcited by 2
  5. 2025Benchmarking Offline Multi-Objective Reinforcement Learning in Critical Carecited by 0
  6. 2026Intervention Granularity Matters: Coherent Treatment Bundles in Counterfactual Simulation with Clinical World Modelscited by 0
  7. 2026Learning response-aware patient dynamics for respiratory supportcited by 0
  8. 2026Lucid Dreaming for World Models: Learning to Doubt Imagination and Decide by Trustcited by 0
  9. 2026PatientWM: Towards Generalizable Clinical World Models via Unified Patient Statescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Unified Benchmark for Dynamic Medical Treatment Reinforcement Learning
  2. 2026Reinforcement Learning over Patient Trajectories for Clinical Reasoning in EHR Foundation Models
  3. 2026EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins
  4. 2024Learning Personalized Prompts for Healthcare Guidance
  5. 2025Reinforcement Learning enhanced Online Adaptive Clinical Decision Support via Digital Twin powered Policy and Treatment Effect optimized Reward
  6. 2024Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimes
  7. 2026The Evidence Ladder for Reinforcement Learning in Healthcare: From Retrospective Policies to Trusted Interventions
  8. 2024Deep Reinforcement Learning for Personalized Diagnostic Decision Pathways Using Electronic Health Records: A Comparative Study on Anemia and Systemic Lupus Erythematosus