mlsanityData to 7 Oct 2026

Guardian-regularized Safe Offline Reinforcement Learning for Smart Weaning of Mechanical Circulatory Devices

arXiv 2511.06111 ↗First posted 8 Nov 2025cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Safe offline RL

Abstract (arXiv)

We study the sequential decision-making problem for automated weaning of mechanical circulatory support (MCS) devices in cardiogenic shock patients. MCS devices are percutaneous micro-axial flow pumps that provide left ventricular unloading and forward blood flow, but current weaning strategies vary significantly across care teams and lack data-driven approaches. Offline reinforcement learning (RL) has proven to be successful in sequential decision-making tasks, but our setting presents challenges for training and evaluating traditional offline RL methods: prohibition of online patient interaction, highly uncertain circulatory dynamics due to concurrent treatments, and limited data availability. We developed an end-to-end machine learning framework with two key contributions (1) Clinically-aware OOD-regularized Model-based Policy Optimization (CORMPO), a density-regularized offline RL algorithm for out-of-distribution suppression that also incorporates clinically-informed reward shaping and (2) a Transformer-based probabilistic digital twin that models MCS circulatory dynamics for policy evaluation with rich physiological and clinical metrics. We prove that textsfCORMPO achieves theoretical performance guarantees under mild assumptions. CORMPO attains a higher reward than the offline RL baselines by 28% and higher scores in clinical metrics by 82.6% on real and synthetic datasets. Our approach offers a principled framework for safe offline policy learning in high-stakes medical applications where domain expertise and safety constraints are essential.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 21 of its 35 references that are in the map. First posted on arXiv 8 Nov 2025.

Builds on

21 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategies22 May

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2017Billion-scale similarity search with GPUscited by 2,477
  5. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  6. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  7. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  8. 2019Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learningcited by 121
  9. 2018Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendationcited by 101
  10. 2022Supported Policy Optimization for Offline Reinforcement Learningcited by 73
  11. 2017Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approachcited by 68
  12. 2017A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Unitscited by 50
8 more
  1. 2023Policy Regularization with Dataset Constraint for Offline Reinforcement Learningcited by 33
  2. 2021Autoregressive Dynamics Models for Offline Policy Evaluation and Optimizationcited by 32
  3. 2022Towards Safe Reinforcement Learning with a Safety Editor Policycited by 28
  4. 2023Constrained Policy Optimization with Explicit Behavior Density for Offline Reinforcement Learningcited by 16
  5. 2022Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learningcited by 14
  6. 2024Med-Real2Sim: Non-Invasive Medical Digital Twins using Physics-Informed Self-Supervised Learningcited by 8
  7. 2024FUSE: Fast Unified Simulation and Estimation for PDEscited by 6
  8. 2024Data-Driven Simulator for Mechanical Circulatory Support with Domain Adversarial Neural Processcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Generative OOD-regularized Model-based Policy Optimizationcited by 0
  2. 2026A Unified Benchmark for Dynamic Medical Treatment Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Ensuring Safety in Automated Mechanical Ventilation through Offline Reinforcement Learning and Digital Twin Verification
  2. 2025Advancing Safe Mechanical Ventilation Using Offline RL With Hybrid Actions and Clinically Aligned Rewards
  3. 2026Offline Reinforcement Learning for Hemodynamic Management of Sepsis in the ICU: a MIMIC-IV Study with Dual Off-Policy Evaluation
  4. 2022Towards Safe Mechanical Ventilation Treatment Using Deep Offline Reinforcement Learning
  5. 2024Offline Inverse Constrained Reinforcement Learning for Safe-Critical Decision Making in Healthcare
  6. 2024Measurement Scheduling for ICU Patients with Offline Reinforcement Learning
  7. 2026medR: Reward Engineering for Clinical Offline Reinforcement Learning via Tri-Drive Potential Functions
  8. 2021Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settings