# Multimodal Prompt Learning with Irregular EHRs for Robust Monitoring of Critical Care Patients

arXiv: https://arxiv.org/abs/2608.21941 · HTML: https://mlsanity.com/paper/2608.21941/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 49 references that are in the map.
- First posted on arXiv 22 Aug 2026.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [MaPLe: Multi-modal Prompt Learning](https://mlsanity.com/paper/2210.03117/) (2022)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Scalable and accurate deep learning for electronic health records](https://mlsanity.com/paper/1801.07860/) (2018)
- [Multi-Time Attention Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/2101.10318/) (2021)
- [Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities](https://mlsanity.com/paper/1812.07809/) (2018)
- [Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis](https://mlsanity.com/paper/1706.03446/) (2017)
- [Set Functions for Time Series](https://mlsanity.com/paper/1909.12064/) (2019)
- [MMTM: Multimodal Transfer Module for CNN Fusion](https://mlsanity.com/paper/1911.08670/) (2019)
- [Multimodal Prompting with Missing Modalities for Visual Recognition](https://mlsanity.com/paper/2303.03369/) (2023)
- [Graph-Guided Network for Irregularly Sampled Multivariate Time Series](https://mlsanity.com/paper/2110.05357/) (2021)
- [Hierarchical Transformers for Long Document Classification](https://mlsanity.com/paper/1910.10781/) (2019)
- [Interpolation-Prediction Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/1909.07782/) (2019)
- [Modeling Missing Data in Clinical Time Series with RNNs](https://mlsanity.com/paper/1606.04130/) (2016)
- [Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequences](https://mlsanity.com/paper/2201.11838/) (2022)
- [MedFuse: Multi-modal fusion with clinical time-series data and chest X-ray images](https://mlsanity.com/paper/2207.07027/) (2022)
- [DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency](https://mlsanity.com/paper/2403.06197/) (2024)
- [Combining 3D Image and Tabular Data via the Dynamic Affine Feature Map Transform](https://mlsanity.com/paper/2107.05990/) (2021)
- [Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modeling](https://mlsanity.com/paper/2210.12156/) (2022)
- [Dynamic Gaussian Mixture based Deep Generative Model For Robust Forecasting on Sparse Multivariate Time Series](https://mlsanity.com/paper/2103.02164/) (2021)
- [Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognition](https://mlsanity.com/paper/2407.05374/) (2024)
- [Missing-modality Enabled Multi-modal Fusion Architecture for Medical Data](https://mlsanity.com/paper/2309.15529/) (2023)
- [SHAPE: A Sample-adaptive Hierarchical Prediction Network for Medication Recommendation](https://mlsanity.com/paper/2309.05675/) (2023)
- [CTPD: Cross-Modal Temporal Pattern Discovery for Enhanced Multimodal Electronic Health Records Analysis](https://mlsanity.com/paper/2411.00696/) (2024)
- [Causal Representation Learning from Multimodal Clinical Records under Non-Random Modality Missingness](https://mlsanity.com/paper/2509.17228/) (2025)

## Abstract (arXiv)

Accurate assessment of patients in intensive care units (ICUs) is essential for timely clinical intervention and improved patient outcomes. Multimodal electronic health records (EHRs), including structured physiological time series and longitudinal clinical notes, provide complementary information for critical care prediction. However, in real-world clinical settings, individual modalities may be partially observed or entirely unavailable, resulting in substantial performance degradation for existing multimodal models. To address this challenge, we propose a multimodal prompt-learning framework for robust clinical prediction under diverse missing-modality scenarios. The proposed framework introduces four complementary types of prompts: generative prompts, missing-signal prompts, missing-type prompts, and temporal prompts. Generative prompts construct surrogate latent representations for unavailable modalities, while missing-signal prompts distinguish observed representations from generated ones. Missing-type prompts condition the model on different modality-availability configurations, whereas temporal prompts perform condition-specific aggregation over temporally encoded clinical sequences. Together, these prompts enable the model to capture missingness-aware intramodal dependencies and cross-modal interactions within a unified architecture. Extensive experiments demonstrate that our method outperforms existing approaches across evaluation metrics on two missingness settings. Ablation and robustness analyses further verify the complementary contributions of the four prompt types and the effectiveness of the proposed framework for clinical prediction from incomplete multimodal EHR data.