RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data
Not on the map. Why?
Abstract (arXiv)
With the improvement of medical data capturing, vast amount of continuous patient monitoring data, e.g., electrocardiogram (ECG), real-time vital signs and medications, become available for clinical decision support at intensive care units (ICUs). However, it becomes increasingly challenging to model such data, due to high density of the monitoring data, heterogeneous data types and the requirement for interpretable models. Integration of these high-density monitoring data with the discrete clinical events (including diagnosis, medications, labs) is challenging but potentially rewarding since richness and granularity in such multimodal data increase the possibilities for accurate detection of complex problems and predicting outcomes (e.g., length of stay and mortality). We propose Recurrent Attentive and Intensive Model (RAIM) for jointly analyzing continuous monitoring data and discrete clinical events. RAIM introduces an efficient attention mechanism for continuous monitoring data (e.g., ECG), which is guided by discrete clinical events (e.g, medication usage). We apply RAIM in predicting physiological decompensation and length of stay in those critically ill patients at ICU. With evaluations on MIMIC- III Waveform Database Matched Subset, we obtain an AUC-ROC score of 90.18% for predicting decompensation and an accuracy of 86.82% for forecasting length of stay with our final model, which outperforms our six baseline models.
Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Jul 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 8; 2022: 10; 2023: 10; 2024: 7; 2025: 11; 2026: 1.
47 papers.
Sort:
By area:
- 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
- 2021Explaining Time Series Predictions with Dynamic Maskscited by 72
- 2022M3Care: Learning with Missing Modalities in Multimodal Healthcare Datacited by 71
- 2023Warpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Seriescited by 54
- 2021MUFASA: Multimodal Fusion Architecture Search for Electronic Health Recordscited by 30
- 2021Temporal Dependencies in Feature Importance for Time Series Predictionscited by 27
- 2021How to Leverage Multimodal EHR Data for Better Medical Predictions?cited by 26
- 2024Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trialscited by 24
- 2025Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictionscited by 18
- 2025Out-of-Distribution Generalization in Time Series: A Surveycited by 15
- 2022Integrating Physiological Time Series and Clinical Notes with Transformer for Early Prediction of Sepsiscited by 14
- 2023Hierarchical Pretraining on Multimodal Electronic Health Recordscited by 11
35 more
- 2024Recent Advances in Predictive Modeling with Electronic Health Recordscited by 11
- 2025Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoringcited by 10
- 2021PyHealth: A Python Library for Health Predictive Modelscited by 9
- 2024SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Predictioncited by 8
- 2023MTS2Graph: Interpretable Multivariate Time Series Classification with Temporal Evolving Graphscited by 7
- 2024Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Modelscited by 7
- 2022Artificial Intelligence for In Silico Clinical Trials: A Reviewcited by 6
- 2022Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Recordscited by 5
- 2023On the Importance of Step-wise Embeddings for Heterogeneous Clinical Time-Seriescited by 5
- 2021Explainable Health Risk Predictor with Transformer-based Medicare Claim Encodercited by 4
- 2022Self-explaining Hierarchical Model for Intraoperative Time Seriescited by 4
- 2021Generalized and Transferable Patient Language Representation for Phenotyping with Limited Datacited by 3
- 2022Multimodal Machine Learning in Precision Healthcited by 3
- 2023Temporal-spatial Correlation Attention Network for Clinical Data Analysis in Intensive Care Unitcited by 3
- 2023PyTrial: Machine Learning Software and Benchmark for Clinical Trial Applicationscited by 3
- 2024Automated Fusion of Multimodal Electronic Health Records for Better Medical Predictionscited by 3
- 2022Self-supervised Representation Learning on Electronic Health Records with Graph Kernel Infomaxcited by 2
- 2024GARNN: An Interpretable Graph Attentive Recurrent Neural Network for Predicting Blood Glucose Levels via Multivariate Time Seriescited by 2
- 2024Large Language Model as a Universal Clinical Multi-task Decodercited by 2
- 2025Representation Learning of Limit Order Book: A Comprehensive Study and Benchmarkingcited by 2
- 2025MEDMKG: Benchmarking Medical Knowledge Exploitation with Multimodal Knowledge Graphcited by 2
- 2025MTM: A Multi-Scale Token Mixing Transformer for Irregular Multivariate Time Series Classificationcited by 2
- 2021Interpretable Additive Recurrent Neural Networks For Multivariate Clinical Time Seriescited by 1
- 2022Label-dependent and event-guided interpretable disease risk prediction using EHRscited by 1
- 2022UnfoldML: Cost-Aware and Uncertainty-Based Dynamic 2D Prediction for Multi-Stage Classificationcited by 1
- 2025Multimodal Disease Progression Modeling via Spatiotemporal Disentanglement and Multiscale Alignmentcited by 1
- 2025CURENet: Combining Unified Representations for Efficient Chronic Disease Predictioncited by 1
- 2026VTBench: A Multimodal Framework for Time-Series Classification with Chart-Based Representationscited by 1
- 2022GRU-TV: Time- and velocity-aware GRU for patient representation on multivariate clinical time-series datacited by 0
- 2023Length of Stay prediction for Hospital Management using Domain Adaptationcited by 0
- 2023Language Model Training Paradigms for Clinical Feature Embeddingscited by 0
- 2023XAI for In-hospital Mortality Prediction via Multimodal ICU Datacited by 0
- 2025Predicting Clinical Outcomes with Waveform LSTMscited by 0
- 2025STRGCN: Capturing Asynchronous Spatio-Temporal Dependencies for Irregular Multivariate Time Series Forecastingcited by 0
- 2025Early GVHD Prediction in Liver Transplantation via Multi-Modal Deep Learning on Imbalanced EHR Datacited by 0
- 2026VTBench: A Multimodal Framework for Time-Series Classification with Chart-Based Representations29 Apr 2026
- 2025CURENet: Combining Unified Representations for Efficient Chronic Disease Prediction14 Nov 2025
- 2025Early GVHD Prediction in Liver Transplantation via Multi-Modal Deep Learning on Imbalanced EHR Data6 Nov 2025
- 2025Multimodal Disease Progression Modeling via Spatiotemporal Disentanglement and Multiscale Alignment13 Oct 2025
- 2025MTM: A Multi-Scale Token Mixing Transformer for Irregular Multivariate Time Series Classification22 Sep 2025
- 2025Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring12 Aug 2025
- 2025Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions30 Jun 2025
- 2025MEDMKG: Benchmarking Medical Knowledge Exploitation with Multimodal Knowledge Graph22 May 2025
- 2025STRGCN: Capturing Asynchronous Spatio-Temporal Dependencies for Irregular Multivariate Time Series Forecasting7 May 2025
- 2025Representation Learning of Limit Order Book: A Comprehensive Study and Benchmarking4 May 2025
- 2025Out-of-Distribution Generalization in Time Series: A Survey18 Mar 2025
- 2025Predicting Clinical Outcomes with Waveform LSTMs13 Mar 2025
35 more
- 2024Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trials6 Sep 2024
- 2024Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models20 Jun 2024
- 2024Large Language Model as a Universal Clinical Multi-task Decoder18 Jun 2024
- 2024SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction15 May 2024
- 2024GARNN: An Interpretable Graph Attentive Recurrent Neural Network for Predicting Blood Glucose Levels via Multivariate Time Series26 Feb 2024
- 2024Recent Advances in Predictive Modeling with Electronic Health Records2 Feb 2024
- 2024Automated Fusion of Multimodal Electronic Health Records for Better Medical Predictions20 Jan 2024
- 2023XAI for In-hospital Mortality Prediction via Multimodal ICU Data29 Dec 2023
- 2023On the Importance of Step-wise Embeddings for Heterogeneous Clinical Time-Series15 Nov 2023
- 2023Language Model Training Paradigms for Clinical Feature Embeddings1 Nov 2023
- 2023Hierarchical Pretraining on Multimodal Electronic Health Records11 Oct 2023
- 2023Length of Stay prediction for Hospital Management using Domain Adaptation29 Jun 2023
- 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects16 Jun 2023
- 2023Warpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Series14 Jun 2023
- 2023PyTrial: Machine Learning Software and Benchmark for Clinical Trial Applications6 Jun 2023
- 2023MTS2Graph: Interpretable Multivariate Time Series Classification with Temporal Evolving Graphs6 Jun 2023
- 2023Temporal-spatial Correlation Attention Network for Clinical Data Analysis in Intensive Care Unit3 Jun 2023
- 2022M3Care: Learning with Missing Modalities in Multimodal Healthcare Data28 Oct 2022
- 2022UnfoldML: Cost-Aware and Uncertainty-Based Dynamic 2D Prediction for Multi-Stage Classification26 Oct 2022
- 2022Self-explaining Hierarchical Model for Intraoperative Time Series10 Oct 2022
- 2022Artificial Intelligence for In Silico Clinical Trials: A Review16 Sep 2022
- 2022Self-supervised Representation Learning on Electronic Health Records with Graph Kernel Infomax1 Sep 2022
- 2022GRU-TV: Time- and velocity-aware GRU for patient representation on multivariate clinical time-series data4 May 2022
- 2022Multimodal Machine Learning in Precision Health10 Apr 2022
- 2022Integrating Physiological Time Series and Clinical Notes with Transformer for Early Prediction of Sepsis28 Mar 2022
- 2022Label-dependent and event-guided interpretable disease risk prediction using EHRs18 Jan 2022
- 2022Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records18 Jan 2022
- 2021How to Leverage Multimodal EHR Data for Better Medical Predictions?29 Oct 2021
- 2021Interpretable Additive Recurrent Neural Networks For Multivariate Clinical Time Series15 Sep 2021
- 2021Temporal Dependencies in Feature Importance for Time Series Predictions29 Jul 2021
- 2021Explaining Time Series Predictions with Dynamic Masks9 Jun 2021
- 2021Explainable Health Risk Predictor with Transformer-based Medicare Claim Encoder19 May 2021
- 2021Generalized and Transferable Patient Language Representation for Phenotyping with Limited Data24 Feb 2021
- 2021MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records3 Feb 2021
- 2021PyHealth: A Python Library for Health Predictive Models11 Jan 2021
- 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
- 2021Explaining Time Series Predictions with Dynamic Maskscited by 72
- 2022M3Care: Learning with Missing Modalities in Multimodal Healthcare Datacited by 71
- 2023Warpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Seriescited by 54
- 2021MUFASA: Multimodal Fusion Architecture Search for Electronic Health Recordscited by 30
- 2021Temporal Dependencies in Feature Importance for Time Series Predictionscited by 27
- 2021How to Leverage Multimodal EHR Data for Better Medical Predictions?cited by 26
- 2024Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trialscited by 24
- 2025Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictionscited by 18
- 2025Out-of-Distribution Generalization in Time Series: A Surveycited by 15
- 2022Integrating Physiological Time Series and Clinical Notes with Transformer for Early Prediction of Sepsiscited by 14
- 2023Hierarchical Pretraining on Multimodal Electronic Health Recordscited by 11
12 most cited of 46 All 46 (JSON)
- 2026VTBench: A Multimodal Framework for Time-Series Classification with Chart-Based Representations29 Apr 2026
- 2025CURENet: Combining Unified Representations for Efficient Chronic Disease Prediction14 Nov 2025
- 2025Early GVHD Prediction in Liver Transplantation via Multi-Modal Deep Learning on Imbalanced EHR Data6 Nov 2025
- 2025Multimodal Disease Progression Modeling via Spatiotemporal Disentanglement and Multiscale Alignment13 Oct 2025
- 2025MTM: A Multi-Scale Token Mixing Transformer for Irregular Multivariate Time Series Classification22 Sep 2025
- 2025Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring12 Aug 2025
- 2025Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions30 Jun 2025
- 2025MEDMKG: Benchmarking Medical Knowledge Exploitation with Multimodal Knowledge Graph22 May 2025
- 2025STRGCN: Capturing Asynchronous Spatio-Temporal Dependencies for Irregular Multivariate Time Series Forecasting7 May 2025
- 2025Representation Learning of Limit Order Book: A Comprehensive Study and Benchmarking4 May 2025
- 2025Out-of-Distribution Generalization in Time Series: A Survey18 Mar 2025
- 2025Predicting Clinical Outcomes with Waveform LSTMs13 Mar 2025
12 newest of 46 All 46 (JSON)