mlsanityData to 14 Oct 2026

Realistic CDSS Drug Dosing with End-to-end Recurrent Q-learning for Dual Vasopressor Control

arXiv 2510.01508 ↗First posted 1 Oct 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for clinical treatment and surgery

Abstract (arXiv)

Reinforcement learning (RL) applications in Clinical Decision Support Systems (CDSS) frequently encounter skepticism because models may recommend inoperable dosing decisions. We propose an end-to-end offline RL framework for dual vasopressor administration in Intensive Care Units (ICUs) that directly addresses this challenge through principled action space design. Our method integrates discrete, continuous, and directional dosing strategies with conservative Q-learning and incorporates a novel recurrent modeling using a replay buffer to capture temporal dependencies in ICU time-series data. Our comparative analysis of norepinephrine dosing strategies across different action space formulations reveals that the designed action spaces improve interpretability and facilitate clinical adoption while preserving efficacy. Empirical results on eICU and MIMIC demonstrate that action space design profoundly influences learned behavioral policies. Compared with baselines, the proposed methods achieve more than 3x expected reward improvements, while aligning with established clinical protocols.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for clinical treatment and surgery, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 27 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

13 of 27 references.

References · 13

  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  5. 2015Prioritized Experience Replaycited by 1,099
  6. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  7. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  8. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  9. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
  10. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 95
  11. 2017Action Branching Architectures for Deep Reinforcement Learningcited by 92
  12. 2019Discrete and Continuous Action Representation for Practical RL in Video Gamescited by 39
1 more
  1. 2022Off-Policy Fitted Q-Evaluation with Differentiable Function Approximators: Z-Estimation and Inference Theorycited by 18

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Offline Reinforcement Learning for Hemodynamic Management of Sepsis in the ICU: a MIMIC-IV Study with Dual Off-Policy Evaluation
  2. 2026On Safer Reinforcement Learning for Sedation and Analgesia in Intensive Care
  3. 2026Insulin4RL: Real-Time Insulin Management in the Intensive Care Unit for Offline Reinforcement Learning
  4. 2023Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care
  5. 2025Learning optimal treatment strategies for intraoperative hypotension using deep reinforcement learning
  6. 2024Development and Validation of Heparin Dosing Policies Using an Offline Reinforcement Learning Algorithm
  7. 2024DTR-Bench: An in silico Environment and Benchmark Platform for Reinforcement Learning Based Dynamic Treatment Regime
  8. 2021Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatment