# Monitoring machine learning (ML)-based risk prediction algorithms in the presence of confounding medical interventions

arXiv: https://arxiv.org/abs/2211.09781 · HTML: https://mlsanity.com/paper/2211.09781/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 59 references that are in the map.
- First posted on arXiv 17 Nov 2022.

## Builds on (most cited first, 6 of 6)

- [Performative Prediction](https://mlsanity.com/paper/2002.06673/) (2020)
- [Bounded, efficient and multiply robust estimation of average treatment effects using instrumental variables](https://mlsanity.com/paper/1611.09925/) (2016)
- [Model updating after interventions paradoxically introduces bias](https://mlsanity.com/paper/2010.11530/) (2020)
- [A likelihood ratio approach to sequential change point detection for a general class of parameters](https://mlsanity.com/paper/1802.07696/) (2018)
- [Bayesian logistic regression for online recalibration and revision of risk prediction models with performance guarantees](https://mlsanity.com/paper/2110.06866/) (2021)
- [Learning (Predictive) Risk Scores in the Presence of Censoring due to Interventions](https://mlsanity.com/paper/1507.07295/) (2015)

## Built on by (most cited first, 2 of 2)

- [Is this model reliable for everyone? Testing for strong calibration](https://mlsanity.com/paper/2307.15247/) (2023)
- [Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lens](https://mlsanity.com/paper/2311.11463/) (2023)

## Abstract (arXiv)

Performance monitoring of machine learning (ML)-based risk prediction models in healthcare is complicated by the issue of confounding medical interventions (CMI): when an algorithm predicts a patient to be at high risk for an adverse event, clinicians are more likely to administer prophylactic treatment and alter the very target that the algorithm aims to predict. A simple approach is to ignore CMI and monitor only the untreated patients, whose outcomes remain unaltered. In general, ignoring CMI may inflate Type I error because (i) untreated patients disproportionally represent those with low predicted risk and (ii) evolution in both the model and clinician trust in the model can induce complex dependencies that violate standard assumptions. Nevertheless, we show that valid inference is still possible if one monitors conditional performance and if either conditional exchangeability or time-constant selection bias hold. Specifically, we develop a new score-based cumulative sum (CUSUM) monitoring procedure with dynamic control limits. Through simulations, we demonstrate the benefits of combining model updating with monitoring and investigate how over-trust in a prediction model may delay detection of performance deterioration. Finally, we illustrate how these monitoring methods can be used to detect calibration decay of an ML-based risk calculator for postoperative nausea and vomiting during the COVID-19 pandemic.