# AI Gone Astray: Technical Supplement

arXiv: https://arxiv.org/abs/2203.16452 · HTML: https://mlsanity.com/paper/2203.16452/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 7 references that are in the map.
- First posted on arXiv 1 Mar 2022.

## Builds on (most cited first, 3 of 3)

- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks](https://mlsanity.com/paper/1908.00690/) (2019)
- [Predicting sepsis in multi-site, multi-national intensive care cohorts using deep learning](https://mlsanity.com/paper/2107.05230/) (2021)

## Abstract (arXiv)

This study is a technical supplement to "AI gone astray: How subtle shifts in patient data send popular algorithms reeling, undermining patient safety." from STAT News, which investigates the effect of time drift on clinically deployed machine learning models. We use MIMIC-IV, a publicly available dataset, to train models that replicate commercial approaches by Dascena and Epic to predict the onset of sepsis, a deadly and yet treatable condition. We observe some of these models degrade overtime; most notably an RNN built on Epic features degrades from a 0.729 AUC to a 0.525 AUC over a decade, leading us to investigate technical and clinical drift as root causes of this performance drop.