# Audit Before You Update: Recording Artifacts and the Limits of Bounded Model Updates in MIMIC-IV ICU Prediction

arXiv: https://arxiv.org/abs/2607.19020 · HTML: https://mlsanity.com/paper/2607.19020/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 43 references that are in the map.
- First posted on arXiv 21 Jul 2026.

## Builds on (most cited first, 15 of 15)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Surgical Fine-Tuning Improves Adaptation to Distribution Shifts](https://mlsanity.com/paper/2210.11466/) (2022)
- [MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval](https://mlsanity.com/paper/2307.00589/) (2023)
- [Almanac: Retrieval-Augmented Language Models for Clinical Medicine](https://mlsanity.com/paper/2303.01229/) (2023)
- [Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks](https://mlsanity.com/paper/1908.00690/) (2019)
- [On Focal Loss for Class-Posterior Probability Estimation: A Theoretical Perspective](https://mlsanity.com/paper/2011.09172/) (2020)
- [A Protocol for Continual Explanation of SHAP](https://mlsanity.com/paper/2306.07218/) (2023)
- [Fair and Explainable Credit-Scoring under Concept Drift: Adaptive Explanation Frameworks for Evolving Populations](https://mlsanity.com/paper/2511.03807/) (2025)

## Abstract (arXiv)

Clinical prediction models in intensive care degrade as data shift, and regulators such as the FDA ask developers to specify bounded update protocols in advance. Evaluations of such protocols assume that apparent drift reflects clinical practice rather than changes in how the database records care. We test whether update strategies can be evaluated on unaudited ICU data, and whether a parameter-bounded update gives up accuracy or attribution stability relative to full fine-tuning once the data are audited. We audit MIMIC-IV 3.1 (2008-2022) for recording artifacts and define a cleaning protocol. On the cleaned cohort, a label-free classifier two-sample test gates updates across three eras. We compare a bounded update, which retrains only treatment-context and fusion parameters, with full fine-tuning for two model families and two targets under a pre-specified non-inferiority margin of 0.015 AUROC, and measure attribution stability against the source model. In 2020-2022 about half of ICU stays have no input or procedure-event records, and a vasopressor ItemID migration makes the 2017-2019 label rate appear to fall when it actually rises. On the cleaned cohort, the gate detects drift in every era, including in physiology-only features, with no false alarms on null data. Bounded updating is non-inferior or superior in 10 of 12 primary cells, but this depends on the target, the model family and learning-rate tuning, and adaptation gains over never updating are small. At a shared learning rate, bounded updates keep physiological attributions more stable in neural models, clearly for vasopressor and less clearly for intubation; with separately tuned rates, full updates are more stable. Audits should precede update evaluation, and the update boundary should be chosen per target.