# Deep Reinforcement Learning for Closed-Loop Blood Glucose Control

arXiv: https://arxiv.org/abs/2009.09051 · HTML: https://mlsanity.com/paper/2009.09051/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2020.

## Built on by (most cited first, 26 of 26)

- [The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models](https://mlsanity.com/paper/2201.03544/) (2022)
- [Correlated Proxies: A New Definition and Improved Mitigation for Reward Hacking](https://mlsanity.com/paper/2403.03185/) (2024)
- [Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetes](https://mlsanity.com/paper/2204.03376/) (2022)
- [DTR-Bench: An in silico Environment and Benchmark Platform for Reinforcement Learning Based Dynamic Treatment Regime](https://mlsanity.com/paper/2405.18610/) (2024)
- [SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer](https://mlsanity.com/paper/2509.18648/) (2025)
- [Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions](https://mlsanity.com/paper/2107.09546/) (2021)
- [Counterfactual-Augmented Importance Sampling for Semi-Offline Policy Evaluation](https://mlsanity.com/paper/2310.17146/) (2023)
- [Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems](https://mlsanity.com/paper/2404.15199/) (2024)
- [ESCADA: Efficient Safety and Context Aware Dose Allocation for Precision Medicine](https://mlsanity.com/paper/2111.13415/) (2021)
- [Forecasting with Sparse but Informative Variables: A Case Study in Predicting Blood Glucose](https://mlsanity.com/paper/2304.08593/) (2023)
- [On the Challenges of using Reinforcement Learning in Precision Drug Dosing: Delay and Prolongedness of Action Effects](https://mlsanity.com/paper/2301.00512/) (2023)
- [The Role of Artificial Intelligence in Enhancing Insulin Recommendations and Therapy Outcomes](https://mlsanity.com/paper/2503.18592/) (2025)
- [Glucose-ML: A collection of longitudinal diabetes datasets for development of robust AI solutions](https://mlsanity.com/paper/2507.14077/) (2025)
- [Repairing Reward Functions with Feedback to Mitigate Reward Hacking](https://mlsanity.com/paper/2510.13036/) (2025)
- [Safety Generalization Under Distribution Shift in Safe Reinforcement Learning: A Diabetes Testbed](https://mlsanity.com/paper/2601.21094/) (2026)
- [Hybrid Control Policy for Artificial Pancreas via Ensemble Deep Reinforcement Learning](https://mlsanity.com/paper/2307.06501/) (2023)
- [Robust Best-arm Identification in Linear Bandits](https://mlsanity.com/paper/2311.04731/) (2023)
- [An Improved Strategy for Blood Glucose Control Using Multi-Step Deep Reinforcement Learning](https://mlsanity.com/paper/2403.07566/) (2024)
- [Personalised Insulin Adjustment with Reinforcement Learning: An In-Silico Validation for People with Diabetes on Intensive Insulin Treatment](https://mlsanity.com/paper/2505.14477/) (2025)
- [Evaluation-Aware Reinforcement Learning](https://mlsanity.com/paper/2509.19464/) (2025)
- [Diabetes Lifestyle Medicine Treatment Assistance Using Reinforcement Learning](https://mlsanity.com/paper/2510.26807/) (2025)
- [GUIDE: Reinforcement Learning for Behavioral Action Support in Type 1 Diabetes](https://mlsanity.com/paper/2604.00385/) (2026)
- [Application of Deep Reinforcement Learning to Event-Triggered Control for Networked Artificial Pancreas Systems](https://mlsanity.com/paper/2604.26126/) (2026)
- [Safe Online Learning via Smooth Safety-Structured Policy Composition](https://mlsanity.com/paper/2606.31320/) (2026)
- [Interpretable Language Model for Closed-Loop Type 1 Diabetes Control](https://mlsanity.com/paper/2607.14126/) (2026)
- [Specifying Reward Functions for RL Without Environment Sampling](https://mlsanity.com/paper/2609.15544/) (2026)

## Abstract (arXiv)

People with type 1 diabetes (T1D) lack the ability to produce the insulin their bodies need. As a result, they must continually make decisions about how much insulin to self-administer to adequately control their blood glucose levels. Longitudinal data streams captured from wearables, like continuous glucose monitors, can help these individuals manage their health, but currently the majority of the decision burden remains on the user. To relieve this burden, researchers are working on closed-loop solutions that combine a continuous glucose monitor and an insulin pump with a control algorithm in an `artificial pancreas.' Such systems aim to estimate and deliver the appropriate amount of insulin. Here, we develop reinforcement learning (RL) techniques for automated blood glucose control. Through a series of experiments, we compare the performance of different deep RL approaches to non-RL approaches. We highlight the flexibility of RL approaches, demonstrating how they can adapt to new individuals with little additional data. On over 2.1 million hours of data from 30 simulated patients, our RL approach outperforms baseline control algorithms: leading to a decrease in median glycemic risk of nearly 50% from 8.34 to 4.24 and a decrease in total time hypoglycemic of 99.8%, from 4,610 days to 6. Moreover, these approaches are able to adapt to predictable meal times (decreasing average risk by an additional 24% as meals increase in predictability). This work demonstrates the potential of deep RL to help people with T1D manage their blood glucose levels without requiring expert knowledge. All of our code is publicly available, allowing for replication and extension.