# Benchmarking machine learning models on multi-centre eICU critical care dataset

arXiv: https://arxiv.org/abs/1910.00964 · HTML: https://mlsanity.com/paper/1910.00964/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2019.

## Built on by (most cited first, 30 of 35)

- [EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models](https://mlsanity.com/paper/2307.02028/) (2023)
- [A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction](https://mlsanity.com/paper/2208.10240/) (2022)
- [An Empirical Framework for Domain Generalization in Clinical Settings](https://mlsanity.com/paper/2103.11163/) (2021)
- [Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine](https://mlsanity.com/paper/2311.04937/) (2023)
- [Algorithm Fairness in AI for Medicine and Healthcare](https://mlsanity.com/paper/2110.00603/) (2021)
- [EHR-R1: A Reasoning-Enhanced Foundational Language Model for Electronic Health Record Analysis](https://mlsanity.com/paper/2510.25628/) (2025)
- [A comparison of approaches to improve worst-case predictive model performance over patient subpopulations](https://mlsanity.com/paper/2108.12250/) (2021)
- [Unmasking the Chameleons: A Benchmark for Out-of-Distribution Detection in Medical Tabular Data](https://mlsanity.com/paper/2309.16220/) (2023)
- [Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clients](https://mlsanity.com/paper/2407.04949/) (2024)
- [One-Versus-Others Attention: Scalable Multimodal Integration for Biomedical Data](https://mlsanity.com/paper/2307.05435/) (2023)
- [Privacy-preserving patient clustering for personalized federated learning](https://mlsanity.com/paper/2307.08847/) (2023)
- [Towards Foundation Models for Critical Care Time Series](https://mlsanity.com/paper/2411.16346/) (2024)
- [Looking for Out-of-Distribution Environments in Multi-center Critical Care Data](https://mlsanity.com/paper/2205.13398/) (2022)
- [CohortNet: Empowering Cohort Discovery for Interpretable Healthcare Analytics](https://mlsanity.com/paper/2406.14015/) (2024)
- [XMI-ICU: Explainable Machine Learning Model for Pseudo-Dynamic Prediction of Mortality in the ICU for Heart Attack Patients](https://mlsanity.com/paper/2305.06109/) (2023)
- [Leveraging MIMIC Datasets for Better Digital Health: A Review on Open Problems, Progress Highlights, and Future Promises](https://mlsanity.com/paper/2506.12808/) (2025)
- [MILM: Large Language Models for Multimodal Irregular Time Series with Informative Sampling](https://mlsanity.com/paper/2605.13711/) (2026)
- [Federated Learning in Multi-Center Critical Care Research: A Systematic Case Study using the eICU Database](https://mlsanity.com/paper/2204.09328/) (2022)
- [Integrated Convolutional and Recurrent Neural Networks for Health Risk Prediction using Patient Journey Data with Many Missing Values](https://mlsanity.com/paper/2211.06045/) (2022)
- [Contrastive Learning-based Imputation-Prediction Networks for In-hospital Mortality Risk Modeling using EHRs](https://mlsanity.com/paper/2308.09896/) (2023)
- [Hypergraph Convolutional Networks for Fine-grained ICU Patient Similarity Analysis and Risk Prediction](https://mlsanity.com/paper/2308.12575/) (2023)
- [CSAI: Conditional Self-Attention Imputation for Healthcare Time-series](https://mlsanity.com/paper/2312.16713/) (2023)
- [Mitigating Overconfidence in Out-of-Distribution Detection by Capturing Extreme Activations](https://mlsanity.com/paper/2405.12658/) (2024)
- [A Multimodal Deep Learning Framework for Predicting ICU Deterioration: Integrating ECG Waveforms with Clinical Data and Clinician Benchmarking](https://mlsanity.com/paper/2601.06645/) (2026)
- [Handling and Interpreting Missing Modalities in Patient Clinical Trajectories via Autoregressive Sequence Modeling](https://mlsanity.com/paper/2604.18753/) (2026)
- [Towards Scheduling Federated Deep Learning using Meta-Gradients for Inter-Hospital Learning](https://mlsanity.com/paper/2107.01707/) (2021)
- [Length of Stay prediction for Hospital Management using Domain Adaptation](https://mlsanity.com/paper/2306.16823/) (2023)
- [P-CAFE: Personalized Cost-Aware Incremental Feature Selection For Electronic Health Records](https://mlsanity.com/paper/2508.08646/) (2025)
- [EMR-AGENT: Automating Cohort and Feature Extraction from EMR Databases](https://mlsanity.com/paper/2510.00549/) (2025)
- [Transfer Learning with Distance Covariance for Random Forest: Error Bounds and an EHR Application](https://mlsanity.com/paper/2510.10870/) (2025)

## Abstract (arXiv)

Progress of machine learning in critical care has been difficult to track, in part due to absence of public benchmarks. Other fields of research (such as computer vision and natural language processing) have established various competitions and public benchmarks. Recent availability of large clinical datasets has enabled the possibility of establishing public benchmarks. Taking advantage of this opportunity, we propose a public benchmark suite to address four areas of critical care, namely mortality prediction, estimation of length of stay, patient phenotyping and risk of decompensation. We define each task and compare the performance of both clinical models as well as baseline and deep learning models using eICU critical care dataset of around 73,000 patients. This is the first public benchmark on a multi-centre critical care dataset, comparing the performance of clinical gold standard with our predictive model. We also investigate the impact of numerical variables as well as handling of categorical variables on each of the defined tasks. The source code, detailing our methods and experiments is publicly available such that anyone can replicate our results and build upon our work.