# End-to-end Offline Reinforcement Learning for Glycemia Control

arXiv: https://arxiv.org/abs/2310.10312 · HTML: https://mlsanity.com/paper/2310.10312/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 13 of its 49 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Placed in

- Offline RL and imitation learning from data / [Safe offline RL](https://mlsanity.com/offlinerl/safe-offline-rl/)

## Builds on (most cited first, 13 of 13)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021)
- [Batch Policy Learning under Constraints](https://mlsanity.com/paper/1903.08738/) (2019)
- [Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetes](https://mlsanity.com/paper/2204.03376/) (2022)
- [Basal Glucose Control in Type 1 Diabetes using Deep Reinforcement Learning: An In Silico Validation](https://mlsanity.com/paper/2005.09059/) (2020)
- [Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applications](https://mlsanity.com/paper/2302.07549/) (2023)
- [No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RL](https://mlsanity.com/paper/2205.08716/) (2022)

## Built on by (most cited first, 2 of 2)

- [The hidden risks of temporal resampling in clinical reinforcement learning](https://mlsanity.com/paper/2602.06603/) (2026)
- [CNA: An AI-Oriented Comprehensive Normalized Assessment for Healthy Status and Application to Optimize RRT Strategies by Reinforcement Learning](https://mlsanity.com/paper/2609.22232/) (2026)

## Abstract (arXiv)

The development of closed-loop systems for glycemia control in type I diabetes relies heavily on simulated patients. Improving the performances and adaptability of these close-loops raises the risk of over-fitting the simulator. This may have dire consequences, especially in unusual cases which were not faithfully-if at all-captured by the simulator. To address this, we propose to use offline RL agents, trained on real patient data, to perform the glycemia control. To further improve the performances, we propose an end-to-end personalization pipeline, which leverages offline-policy evaluation methods to remove altogether the need of a simulator, while still enabling an estimation of clinically relevant metrics for diabetes.