mlsanityData to 7 Oct 2026

Insulin4RL: Real-Time Insulin Management in the Intensive Care Unit for Offline Reinforcement Learning

arXiv 2606.19481 ↗First posted 17 Jun 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Safe offline RL

Abstract (arXiv)

Offline reinforcement learning (ORL) offers the potential to improve the quality of clinical decision-making using historical electronic health record (EHR) data. Current training and evaluative practices in this field rely heavily on EHR datasets that have been temporally discretised into fixed, regular time intervals. Discretisation creates fictional representations of complex clinical scenarios and compromises the generalisability of retrospective model evaluations. In this paper, we introduce Insulin4RL, a healthcare ORL dataset featuring naturally irregular inputs and actions from real clinical trajectories. Derived from MIMIC-IV, Insulin4RL comprises over 375,000 labelled decisions across 12,209 patients requiring insulin infusion titration in the Intensive Care Unit. The dataset can thus be used for research into ORL model performance under realistic clinical sampling assumptions. We provide a description of the dataset's structure and characteristics, baseline performance metrics using model-free offline reinforcement learning, and a standardised evaluation protocol using fitted Q-evaluation. We conclude with suggested areas for future research that could be addressed using this resource.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 62 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

26 of 62 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026The hidden risks of temporal resampling in clinical reinforcement learning6 Feb
  2. 2025Exploring Time-Step Size in Reinforcement Learning for Sepsis Treatment25 Nov
  3. 2021Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settings23 Jul
  4. 2021Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatment9 Jan

Other references · 22

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  4. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  5. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  6. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 337
  7. 2019Batch Policy Learning under Constraintscited by 242
  8. 2018Deep Variational Reinforcement Learning for POMDPscited by 122
  9. 2019Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learningcited by 121
  10. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  11. 2021Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Seriescited by 96
  12. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 86
10 more
  1. 2017Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approachcited by 68
  2. 2018Evaluating Reinforcement Learning Algorithms in Observational Health Settingscited by 60
  3. 2019Making Deep Q-learning methods robust to time discretizationcited by 51
  4. 2021Medical Dead-ends and Learning to Identify High-risk States and Treatmentscited by 28
  5. 2020Clinician-in-the-Loop Decision Making: Reinforcement Learning with Near-Optimal Set-Valued Policiescited by 24
  6. 2022The Health Gym: Synthetic Health-Related Datasets for the Development of Reinforcement Learning Algorithmscited by 19
  7. 2019Does the "Artificial Intelligence Clinician" learn optimal treatment strategies for sepsis in intensive care?cited by 9
  8. 2023Offline Learning of Closed-Loop Deep Brain Stimulation Controllers for Parkinson Disease Treatmentcited by 8
  9. 2018Discretizing Logged Interaction Data Biases Learning for Decision-Makingcited by 3
  10. 2025MIMIC-Sepsis: A Curated Benchmark for Modeling and Learning from Sepsis Trajectories in the ICUcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetes
  2. 2025Flexible Blood Glucose Control: Offline Reinforcement Learning from Human Feedback
  3. 2024Measurement Scheduling for ICU Patients with Offline Reinforcement Learning
  4. 2026GUIDE: Reinforcement Learning for Behavioral Action Support in Type 1 Diabetes
  5. 2023Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applications
  6. 2023End-to-end Offline Reinforcement Learning for Glycemia Control
  7. 2025Realistic CDSS Drug Dosing with End-to-end Recurrent Q-learning for Dual Vasopressor Control
  8. 2025Are Large Language Models Dynamic Treatment Planners? An In Silico Study from a Prior Knowledge Injection Angle