# An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness

arXiv: https://arxiv.org/abs/2604.23954 · HTML: https://mlsanity.com/paper/2604.23954/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 27 Apr 2026.

## Builds on (most cited first, 15 of 15)

- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification](https://mlsanity.com/paper/2301.11562/) (2023)
- [Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learning](https://mlsanity.com/paper/2501.14959/) (2025)
- [The Data Addition Dilemma](https://mlsanity.com/paper/2408.04154/) (2024)
- [Exploration of the Rashomon Set Assists Trustworthy Explanations for Medical Data](https://mlsanity.com/paper/2308.11446/) (2023)
- [Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set](https://mlsanity.com/paper/2501.15634/) (2025)
- [Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments](https://mlsanity.com/paper/2411.00186/) (2024)
- [When to retrain a machine learning model](https://mlsanity.com/paper/2505.14903/) (2025)
- [Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradation](https://mlsanity.com/paper/2506.17442/) (2025)
- [Allocation Multiplicity: Evaluating the Promises of the Rashomon Set](https://mlsanity.com/paper/2503.16621/) (2025)
- [A moving target in AI-assisted decision-making: Dataset shift, model updating, and the problem of update opacity](https://mlsanity.com/paper/2504.05210/) (2025)
- [Towards Stable Machine Learning Model Retraining via Slowly Varying Sequences](https://mlsanity.com/paper/2403.19871/) (2024)
- [Fairness Evolution in Continual Learning for Medical Imaging](https://mlsanity.com/paper/2406.02480/) (2024)
- [Regulating AI Adaptation: An Analysis of AI Medical Device Updates](https://mlsanity.com/paper/2407.16900/) (2024)

## Abstract (arXiv)

Artificial Intelligence (AI) and Machine Learning (ML) models used in clinical settings are increasingly deployed to support clinical decision-making. However, when training data become stale due to changes in demographics, environment, or patient behaviors, model performance can degrade substantially. While updating models with new training data is necessary, such updates may also introduce new risks. We evaluated the proposed monitoring framework on four publicly available U.S.-based Type 1 Diabetes datasets containing high-resolution continuous glucose monitoring (CGM) data, comprising approximately 11,300 weekly observations from 496 participants younger than 20 years. All datasets included structured sociodemographic information. Using the prediction of severe hyperglycemia events in children with Type 1 Diabetes as a case study, we examine how different model update strategies can adversely affect model stability by causing predictions to change for a large number of cases after retraining, increase prediction arbitrariness, and worsen subgroup fairness and the balance of error rates across populations. We propose multiple dimensions for continuous monitoring to detect these issues and argue that such monitoring is essential for the development of trustworthy clinical decision support systems.