mlsanityData to 14 Oct 2026

Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetes

arXiv 2204.03376 ↗First posted 7 Apr 2022cs.LG cs.AIcited by 23

On the map: Offline RL and imitation learning from data → Safe offline RL

Abstract (arXiv)

The widespread adoption of effective hybrid closed loop systems would represent an important milestone of care for people living with type 1 diabetes (T1D). These devices typically utilise simple control algorithms to select the optimal insulin dose for maintaining blood glucose levels within a healthy range. Online reinforcement learning (RL) has been utilised as a method for further enhancing glucose control in these devices. Previous approaches have been shown to reduce patient risk and improve time spent in the target range when compared to classical control algorithms, but are prone to instability in the learning process, often resulting in the selection of unsafe actions. This work presents an evaluation of offline RL for developing effective dosing policies without the need for potentially dangerous patient interaction during training. This paper examines the utility of BCQ, CQL and TD3-BC in managing the blood glucose of the 30 virtual patients available within the FDA-approved UVA/Padova glucose dynamics simulator. When trained on less than a tenth of the total training samples required by online RL to achieve stable performance, this work shows that offline RL can significantly increase time in the healthy blood glucose range from 61.6 +- 0.3% to 65.3 +/- 0.5% when compared to the strongest state-of-art baseline (p < 0.001). This is achieved without any associated increase in low blood glucose events. Offline RL is also shown to be able to correct for common and challenging control scenarios such as incorrect bolus dosing, irregular meal timings and compression errors.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 74 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

16 of 74 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settings23 Jul
  2. 2021Offline reinforcement learning with uncertainty for treatment strategies in sepsis9 Jul

Other references · 14

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  7. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  8. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  9. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 148
  10. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  11. 2020Deep Reinforcement Learning for Closed-Loop Blood Glucose Controlcited by 26
  12. 2020Basal Glucose Control in Type 1 Diabetes using Deep Reinforcement Learning: An In Silico Validationcited by 18
2 more
  1. 2022Statistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizonscited by 10
  2. 2019A Dual-Hormone Closed-Loop Delivery System for Type 1 Diabetes Using Deep Reinforcement Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 8; 2025: 8; 2026: 3.

23 papers.

Sort:

By area:

  1. 2024TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agentscited by 7
  2. 2026MetaboNet: The Largest Publicly Available Consolidated Dataset for Type 1 Diabetes Managementcited by 5
  3. 2023Generating Personalized Insulin Treatments Strategies with Deep Conditional Generative Time Series Modelscited by 4
  4. 2024Mitigating Deep Reinforcement Learning Backdoors in the Neural Activation Spacecited by 4
  5. 2024Offline Reinforcement Learning with Behavioral Supervisor Tuningcited by 3
  6. 2024ADESSE: Advice Explanations in Complex Repeated Decision-Making Environmentscited by 3
  7. 2023Towards Real-World Applications of Personalized Anesthesia Using Policy Constraint Q Learning for Propofol Infusion Controlcited by 2
  8. 2023End-to-end Offline Reinforcement Learning for Glycemia Controlcited by 2
  9. 2023Basal-Bolus Advisor for Type 1 Diabetes (T1D) Patients Using Multi-Agent Reinforcement Learning (RL) Methodologycited by 1
  10. 2024Methodology for Interpretable Reinforcement Learning for Optimizing Mechanical Ventilationcited by 1
  11. 2024Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbationscited by 1
  12. 2025The Role of Artificial Intelligence in Enhancing Insulin Recommendations and Therapy Outcomescited by 1
11 more
  1. 2025ExOSITO: Explainable Off-Policy Learning with Side Information for Intensive Care Unit Blood Test Orderscited by 1
  2. 2025PrivORL: Differentially Private Synthetic Dataset for Offline Reinforcement Learningcited by 1
  3. 2024An Improved Strategy for Blood Glucose Control Using Multi-Step Deep Reinforcement Learningcited by 0
  4. 2024Blood Glucose Control Via Pre-trained Counterfactual Invertible Neural Networkscited by 0
  5. 2025Dual Alignment Maximin Optimization for Offline Model-based RLcited by 0
  6. 2025Personalised Insulin Adjustment with Reinforcement Learning: An In-Silico Validation for People with Diabetes on Intensive Insulin Treatmentcited by 0
  7. 2025Towards an Adaptive Social Game-Playing Robot: An Offline Reinforcement Learning-Based Frameworkcited by 0
  8. 2025Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot Controlcited by 0
  9. 2025Diabetes Lifestyle Medicine Treatment Assistance Using Reinforcement Learningcited by 0
  10. 2026GUIDE: Reinforcement Learning for Behavioral Action Support in Type 1 Diabetescited by 0
  11. 2026PhiBE-Q-Learning: Bridging Off-Policy Reinforcement Learning and Continuous-Time Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Flexible Blood Glucose Control: Offline Reinforcement Learning from Human Feedback
  2. 2026Insulin4RL: Real-Time Insulin Management in the Intensive Care Unit for Offline Reinforcement Learning
  3. 2023Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applications
  4. 2025Integrating Neural Differential Forecasting with Safe Reinforcement Learning for Blood Glucose Regulation
  5. 2026Interpretable Language Model for Closed-Loop Type 1 Diabetes Control
  6. 2026Learning to Compress Time-to-Control: A Reinforcement Learning Framework for Chronic Disease Management
  7. 2026Safety Generalization Under Distribution Shift in Safe Reinforcement Learning: A Diabetes Testbed
  8. 2026The hidden risks of temporal resampling in clinical reinforcement learning