Safety Generalization Under Distribution Shift in Safe Reinforcement Learning: A Diabetes Testbed
Not on the map. Why?
Abstract (arXiv)
Safe Reinforcement Learning (RL) algorithms are typically evaluated under fixed training conditions. We investigate whether training-time safety guarantees transfer to deployment under distribution shift, using diabetes management as a safety-critical testbed. We benchmark safe RL algorithms on a unified clinical simulator and reveal a safety generalization gap: policies satisfying constraints during training frequently violate safety requirements on unseen patients. We demonstrate that test-time shielding, which filters unsafe actions using learned dynamics models, effectively restores safety across algorithms and patient populations. Across eight safe RL algorithms, three diabetes types, and three age groups, shielding achieves Time-in-Range gains of 13--14% for strong baselines such as PPO-Lag and CPO while reducing clinical risk index and glucose variability. Our simulator and benchmark provide a platform for studying safety under distribution shift in safety-critical control domains. Code is available at https://github.com/safe-autonomy-lab/GlucoSim and https://github.com/safe-autonomy-lab/GlucoAlg.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 70 references that are in the map. First posted on arXiv 28 Jan 2026.
Builds on
29 of 70 references.
References · 29
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2015Trust Region Policy Optimizationcited by 2,503
- 2016Concrete Problems in AI Safetycited by 1,582
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
- 2017Constrained Policy Optimizationcited by 928
- 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
- 2017Robust Adversarial Reinforcement Learningcited by 439
- 2017Safe Reinforcement Learning via Shieldingcited by 427
- 2018Reward Constrained Policy Optimizationcited by 397
- 2018Quantifying Generalization in Reinforcement Learningcited by 345
- 2020Projection-Based Constrained Policy Optimizationcited by 195
17 more
- 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
- 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
- 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 128
- 2019Observational Overfitting in Reinforcement Learningcited by 90
- 2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 76
- 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 55
- 2022Safe Reinforcement Learning via Shielding under Partial Observabilitycited by 49
- 2023Safe Reinforcement Learning via Probabilistic Logic Shieldscited by 41
- 2020Deep Reinforcement Learning for Closed-Loop Blood Glucose Controlcited by 26
- 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learningcited by 22
- 2024Zero-Shot Reinforcement Learning via Function Encoderscited by 17
- 2024A CMDP-within-online framework for Meta-Safe Reinforcement Learningcited by 12
- 2025Function Encoders: A Principled Approach to Transfer Learning in Hilbert Spacescited by 9
- 2023Safe POMDP Online Planning via Shieldingcited by 8
- 2024Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulationcited by 8
- 2024Safe POMDP Online Planning among Dynamic Agents via Adaptive Conformal Predictioncited by 6
- 2025Conformal Safety Shielding for Imperfect-Perception Agentscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.