# A Multimodal Deep Learning Framework for Predicting ICU Deterioration: Integrating ECG Waveforms with Clinical Data and Clinician Benchmarking

arXiv: https://arxiv.org/abs/2601.06645 · HTML: https://mlsanity.com/paper/2601.06645/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 10 Jan 2026.

## Builds on (most cited first, 16 of 16)

- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [TabArena: A Living Benchmark for Machine Learning on Tabular Data](https://mlsanity.com/paper/2506.16791/) (2025)
- [When combinations of humans and AI are useful: A systematic review and meta-analysis](https://mlsanity.com/paper/2405.06087/) (2024)
- [EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models](https://mlsanity.com/paper/2307.02028/) (2023)
- [Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data](https://mlsanity.com/paper/2407.04491/) (2024)
- [A Closer Look at AUROC and AUPRC under Class Imbalance](https://mlsanity.com/paper/2401.06091/) (2024)
- [An Extensive Data Processing Pipeline for MIMIC-IV](https://mlsanity.com/paper/2204.13841/) (2022)
- [HiRID-ICU-Benchmark -- A Comprehensive Machine Learning Benchmark on High-resolution ICU Data](https://mlsanity.com/paper/2111.08536/) (2021)
- [Benchmarking machine learning models on multi-centre eICU critical care dataset](https://mlsanity.com/paper/1910.00964/) (2019)
- [Yet Another ICU Benchmark: A Flexible Multi-Center Framework for Clinical ML](https://mlsanity.com/paper/2306.05109/) (2023)
- [Towards quantitative precision for ECG analysis: Leveraging state space models, self-supervision and patient metadata](https://mlsanity.com/paper/2308.15291/) (2023)
- [Enhancing clinical decision support with physiological waveforms -- a multimodal benchmark in emergency care](https://mlsanity.com/paper/2407.17856/) (2024)
- [Imputation for prediction: beware of diminishing returns](https://mlsanity.com/paper/2407.19804/) (2024)
- [Benchmarking ECG FMs: A Reality Check Across Clinical Tasks](https://mlsanity.com/paper/2509.25095/) (2025)
- [Explaining Time Series Classification Predictions via Causal Attributions](https://mlsanity.com/paper/2405.15871/) (2024)
- [Towards actionable hypotension prediction -- predicting catecholamine therapy initiation in the intensive care unit](https://mlsanity.com/paper/2510.24287/) (2025)

## Built on by (most cited first, 1 of 1)

- [Explainable Diabetic Retinopathy Classification Using Vision Foundation Models](https://mlsanity.com/paper/2608.28207/) (2026)

## Abstract (arXiv)

Artificial intelligence holds strong potential to support clinical decision making in intensive care units where timely and accurate risk assessment is critical. However, many existing models focus on isolated outcomes or limited data types, while clinicians integrate longitudinal history, real time physiology, and heterogeneous clinical information. To address this gap, we developed MDS ICU, a unified multimodal machine learning framework that fuses routinely collected data including demographics, biometrics, vital signs, laboratory values, ECG waveforms, surgical procedures, and medical device usage to provide continuous predictive support during ICU stays. Using 63001 samples from 27062 patients in MIMIC IV, we trained a deep learning architecture that combines structured state space S4 encoders for ECG waveforms with multilayer perceptron RealMLP encoders for tabular data to jointly predict 33 clinically relevant outcomes spanning mortality, organ dysfunction, medication needs, and acute deterioration. The model achieved strong discrimination with AUROCs of 0.90 for 24 hour mortality, 0.92 for sedative administration, 0.97 for invasive mechanical ventilation, and 0.93 for coagulation dysfunction. Calibration analysis showed close agreement between predicted and observed risks, with consistent gains from ECG waveform integration. Comparisons with clinicians and large language models showed that model predictions alone outperformed both, and that providing model outputs as decision support further improved their performance. These results demonstrate that multimodal AI can deliver clinically meaningful risk stratification across diverse ICU outcomes while augmenting rather than replacing clinical expertise, establishing a scalable foundation for precision critical care decision support.