# Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lens

arXiv: https://arxiv.org/abs/2311.11463 · HTML: https://mlsanity.com/paper/2311.11463/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 62 references that are in the map.
- First posted on arXiv 20 Nov 2023.

## Builds on (most cited first, 18 of 18)

- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [Performative Prediction](https://mlsanity.com/paper/2002.06673/) (2020)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](https://mlsanity.com/paper/1710.11214/) (2017)
- [Safe Testing](https://mlsanity.com/paper/1906.07801/) (2019)
- [Graphical Models for Processing Missing Data](https://mlsanity.com/paper/1801.03583/) (2018)
- [Anticipating Performativity by Predicting from Predictions](https://mlsanity.com/paper/2208.07331/) (2022)
- [On the Opportunity of Causal Learning in Recommendation Systems: Foundation, Estimation, Prediction and Challenges](https://mlsanity.com/paper/2201.06716/) (2022)
- [Monitoring and explainability of models in production](https://mlsanity.com/paper/2007.06299/) (2020)
- [Benchmarking Deep Learning Architectures for Predicting Readmission to the ICU and Describing Patients-at-Risk](https://mlsanity.com/paper/1905.08547/) (2019)
- [Model updating after interventions paradoxically introduces bias](https://mlsanity.com/paper/2010.11530/) (2020)
- [A likelihood ratio approach to sequential change point detection for a general class of parameters](https://mlsanity.com/paper/1802.07696/) (2018)
- [Is this model reliable for everyone? Testing for strong calibration](https://mlsanity.com/paper/2307.15247/) (2023)
- [DEPLOYR: A technical framework for deploying custom real-time machine learning models into the electronic medical record](https://mlsanity.com/paper/2303.06269/) (2023)
- [Learning (Predictive) Risk Scores in the Presence of Censoring due to Interventions](https://mlsanity.com/paper/1507.07295/) (2015)
- [Monitoring machine learning (ML)-based risk prediction algorithms in the presence of confounding medical interventions](https://mlsanity.com/paper/2211.09781/) (2022)

## Built on by (most cited first, 3 of 3)

- [Automatic dataset shift identification to support safe deployment of medical imaging AI](https://mlsanity.com/paper/2411.07940/) (2024)
- [Reliably Detecting Model Failures in Deployment Without Labels](https://mlsanity.com/paper/2506.05047/) (2025)
- [When Predictions Shape Reality: A Socio-Technical Synthesis of Performative Predictions in Machine Learning](https://mlsanity.com/paper/2601.04447/) (2026)

## Abstract (arXiv)

After a machine learning (ML)-based system is deployed, monitoring its performance is important to ensure the safety and effectiveness of the algorithm over time. When an ML algorithm interacts with its environment, the algorithm can affect the data-generating mechanism and be a major source of bias when evaluating its standalone performance, an issue known as performativity. Although prior work has shown how to validate models in the presence of performativity using causal inference techniques, there has been little work on how to monitor models in the presence of performativity. Unlike the setting of model validation, there is much less agreement on which performance metrics to monitor. Different monitoring criteria impact how interpretable the resulting test statistic is, what assumptions are needed for identifiability, and the speed of detection. When this choice is further coupled with the decision to use observational versus interventional data, ML deployment teams are faced with a multitude of monitoring options. The aim of this work is to highlight the relatively under-appreciated complexity of designing a monitoring strategy and how causal reasoning can provide a systematic framework for choosing between these options. As a motivating example, we consider an ML-based risk prediction algorithm for predicting unplanned readmissions. Bringing together tools from causal inference and statistical process control, we consider six monitoring procedures (three candidate monitoring criteria and two data sources) and investigate their operating characteristics in simulation studies. Results from this case study emphasize the seemingly simple (and obvious) fact that not all monitoring systems are created equal, which has real-world impacts on the design and documentation of ML monitoring systems.