# Rethinking clinical prediction: Why machine learning must consider year of care and feature aggregation

arXiv: https://arxiv.org/abs/1811.12583 · HTML: https://mlsanity.com/paper/1811.12583/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2018.

## Built on by (most cited first, 4 of 4)

- [AutoPrognosis 2.0: Democratizing Diagnostic and Prognostic Modeling in Healthcare with Automated Machine Learning](https://mlsanity.com/paper/2210.12090/) (2022)
- [Mind the Performance Gap: Examining Dataset Shift During Prospective Validation](https://mlsanity.com/paper/2107.13964/) (2021)
- [Improving generalization of machine learning-identified biomarkers with causal modeling: an investigation into immune receptor diagnostics](https://mlsanity.com/paper/2204.09291/) (2022)
- [Deployment of a Robust and Explainable Mortality Prediction Model: The COVID-19 Pandemic and Beyond](https://mlsanity.com/paper/2311.17133/) (2023)

## Abstract (arXiv)

Machine learning for healthcare often trains models on de-identified datasets with randomly-shifted calendar dates, ignoring the fact that data were generated under hospital operation practices that change over time. These changing practices induce definitive changes in observed data which confound evaluations which do not account for dates and limit the generalisability of date-agnostic models. In this work, we establish the magnitude of this problem on MIMIC, a public hospital dataset, and showcase a simple solution. We augment MIMIC with the year in which care was provided and show that a model trained using standard feature representations will significantly degrade in quality over time. We find a deterioration of 0.3 AUC when evaluating mortality prediction on data from 10 years later. We find a similar deterioration of 0.15 AUC for length-of-stay. In contrast, we demonstrate that clinically-oriented aggregates of raw features significantly mitigate future deterioration. Our suggested aggregated representations, when retrained yearly, have prediction quality comparable to year-agnostic models.