# RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data

arXiv: https://arxiv.org/abs/1807.08820 · HTML: https://mlsanity.com/paper/1807.08820/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2018.

## Built on by (most cited first, 30 of 47)

- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [Explaining Time Series Predictions with Dynamic Masks](https://mlsanity.com/paper/2106.05303/) (2021)
- [M3Care: Learning with Missing Modalities in Multimodal Healthcare Data](https://mlsanity.com/paper/2210.17292/) (2022)
- [Warpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Series](https://mlsanity.com/paper/2306.09368/) (2023)
- [MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records](https://mlsanity.com/paper/2102.02340/) (2021)
- [Temporal Dependencies in Feature Importance for Time Series Predictions](https://mlsanity.com/paper/2107.14317/) (2021)
- [How to Leverage Multimodal EHR Data for Better Medical Predictions?](https://mlsanity.com/paper/2110.15763/) (2021)
- [Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trials](https://mlsanity.com/paper/2409.04481/) (2024)
- [Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions](https://mlsanity.com/paper/2507.00191/) (2025)
- [Out-of-Distribution Generalization in Time Series: A Survey](https://mlsanity.com/paper/2503.13868/) (2025)
- [Integrating Physiological Time Series and Clinical Notes with Transformer for Early Prediction of Sepsis](https://mlsanity.com/paper/2203.14469/) (2022)
- [Hierarchical Pretraining on Multimodal Electronic Health Records](https://mlsanity.com/paper/2310.07871/) (2023)
- [Recent Advances in Predictive Modeling with Electronic Health Records](https://mlsanity.com/paper/2402.01077/) (2024)
- [Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring](https://mlsanity.com/paper/2508.09085/) (2025)
- [PyHealth: A Python Library for Health Predictive Models](https://mlsanity.com/paper/2101.04209/) (2021)
- [SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction](https://mlsanity.com/paper/2405.09039/) (2024)
- [MTS2Graph: Interpretable Multivariate Time Series Classification with Temporal Evolving Graphs](https://mlsanity.com/paper/2306.03834/) (2023)
- [Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models](https://mlsanity.com/paper/2406.13942/) (2024)
- [Artificial Intelligence for In Silico Clinical Trials: A Review](https://mlsanity.com/paper/2209.09023/) (2022)
- [Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records](https://mlsanity.com/paper/2201.06779/) (2022)
- [On the Importance of Step-wise Embeddings for Heterogeneous Clinical Time-Series](https://mlsanity.com/paper/2311.08902/) (2023)
- [Explainable Health Risk Predictor with Transformer-based Medicare Claim Encoder](https://mlsanity.com/paper/2105.09428/) (2021)
- [Self-explaining Hierarchical Model for Intraoperative Time Series](https://mlsanity.com/paper/2210.04417/) (2022)
- [Generalized and Transferable Patient Language Representation for Phenotyping with Limited Data](https://mlsanity.com/paper/2103.00482/) (2021)
- [Multimodal Machine Learning in Precision Health](https://mlsanity.com/paper/2204.04777/) (2022)
- [Temporal-spatial Correlation Attention Network for Clinical Data Analysis in Intensive Care Unit](https://mlsanity.com/paper/2306.01970/) (2023)
- [PyTrial: Machine Learning Software and Benchmark for Clinical Trial Applications](https://mlsanity.com/paper/2306.04018/) (2023)
- [Automated Fusion of Multimodal Electronic Health Records for Better Medical Predictions](https://mlsanity.com/paper/2401.11252/) (2024)
- [Self-supervised Representation Learning on Electronic Health Records with Graph Kernel Infomax](https://mlsanity.com/paper/2209.00655/) (2022)
- [GARNN: An Interpretable Graph Attentive Recurrent Neural Network for Predicting Blood Glucose Levels via Multivariate Time Series](https://mlsanity.com/paper/2402.16230/) (2024)

## Abstract (arXiv)

With the improvement of medical data capturing, vast amount of continuous patient monitoring data, e.g., electrocardiogram (ECG), real-time vital signs and medications, become available for clinical decision support at intensive care units (ICUs). However, it becomes increasingly challenging to model such data, due to high density of the monitoring data, heterogeneous data types and the requirement for interpretable models. Integration of these high-density monitoring data with the discrete clinical events (including diagnosis, medications, labs) is challenging but potentially rewarding since richness and granularity in such multimodal data increase the possibilities for accurate detection of complex problems and predicting outcomes (e.g., length of stay and mortality). We propose Recurrent Attentive and Intensive Model (RAIM) for jointly analyzing continuous monitoring data and discrete clinical events. RAIM introduces an efficient attention mechanism for continuous monitoring data (e.g., ECG), which is guided by discrete clinical events (e.g, medication usage). We apply RAIM in predicting physiological decompensation and length of stay in those critically ill patients at ICU. With evaluations on MIMIC- III Waveform Database Matched Subset, we obtain an AUC-ROC score of 90.18% for predicting decompensation and an accuracy of 86.82% for forecasting length of stay with our final model, which outperforms our six baseline models.