# SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction

arXiv: https://arxiv.org/abs/2405.09039 · HTML: https://mlsanity.com/paper/2405.09039/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 67 references that are in the map.
- First posted on arXiv 15 May 2024.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism](https://mlsanity.com/paper/1608.05745/) (2016)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Multi-Time Attention Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/2101.10318/) (2021)
- [GRAM: Graph-based Attention Model for Healthcare Representation Learning](https://mlsanity.com/paper/1611.07012/) (2016)
- [Set Functions for Time Series](https://mlsanity.com/paper/1909.12064/) (2019)
- [Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks](https://mlsanity.com/paper/1706.05764/) (2017)
- [Graph-Guided Network for Irregularly Sampled Multivariate Time Series](https://mlsanity.com/paper/2110.05357/) (2021)
- [Attend and Diagnose: Clinical Time Series Analysis using Attention Models](https://mlsanity.com/paper/1711.03905/) (2017)
- [Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences](https://mlsanity.com/paper/1610.09513/) (2016)
- [Interpolation-Prediction Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/1909.07782/) (2019)
- [M3Care: Learning with Missing Modalities in Multimodal Healthcare Data](https://mlsanity.com/paper/2210.17292/) (2022)
- [ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling](https://mlsanity.com/paper/2402.10635/) (2024)
- [GraphCare: Enhancing Healthcare Predictions with Personalized Knowledge Graphs](https://mlsanity.com/paper/2305.12788/) (2023)
- [ConCare: Personalized Clinical Feature Embedding via Capturing the Healthcare Context](https://mlsanity.com/paper/1911.12216/) (2019)
- [Warpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Series](https://mlsanity.com/paper/2306.09368/) (2023)
- [AdaCare: Explainable Clinical Health Status Representation Learning via Scale-Adaptive Feature Extraction and Recalibration](https://mlsanity.com/paper/1911.12205/) (2019)
- [StageNet: Stage-Aware Neural Networks for Health Risk Prediction](https://mlsanity.com/paper/2001.10054/) (2020)
- [RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data](https://mlsanity.com/paper/1807.08820/) (2018)
- [Deep Representation Learning of Patient Data from Electronic Health Records (EHR): A Systematic Review](https://mlsanity.com/paper/2010.02809/) (2020)
- [Deep Representation Learning of Electronic Health Records to Unlock Patient Stratification at Scale](https://mlsanity.com/paper/2003.06516/) (2020)
- [Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modeling](https://mlsanity.com/paper/2210.12156/) (2022)
- [Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record](https://mlsanity.com/paper/1810.04793/) (2018)
- [Collaborative Graph Learning with Auxiliary Text for Temporal Event Prediction in Healthcare](https://mlsanity.com/paper/2105.07542/) (2021)

## Built on by (most cited first, 8 of 8)

- [GlucoFM: A Dual-Stream Foundation Model for Continuous Glucose Monitoring](https://mlsanity.com/paper/2605.30865/) (2026)
- [Bridging Data Gaps of Rare Conditions in ICU: A Multi-Disease Adaptation Approach for Clinical Prediction](https://mlsanity.com/paper/2507.06432/) (2025)
- [Diffmv: A Unified Diffusion Framework for Healthcare Predictions with Random Missing Views and View Laziness](https://mlsanity.com/paper/2505.11802/) (2025)
- [Improving Hospital Risk Prediction with Knowledge-Augmented Multimodal EHR Modeling](https://mlsanity.com/paper/2508.01970/) (2025)
- [Think as a Doctor: An Interpretable AI Approach for ICU Mortality Prediction](https://mlsanity.com/paper/2510.11745/) (2025)
- [Structure-Aware Set Transformers: Temporal and Variable-Type Attention Biases for Asynchronous Clinical Time Series](https://mlsanity.com/paper/2603.06605/) (2026)
- [Dementia Etiology Diagnosis via Collaborative Meta Knowledge Enhancement](https://mlsanity.com/paper/2607.22770/) (2026)
- [NeuroECG: ECGFounder-Based Deep ECG Representation for EEG-Free Neurological Prognostication After Cardiac Arrest](https://mlsanity.com/paper/2609.18891/) (2026)

## Abstract (arXiv)

Electronic health record (EHR) data has emerged as a valuable resource for analyzing patient health status. However, the prevalence of missing data in EHR poses significant challenges to existing methods, leading to spurious correlations and suboptimal predictions. While various imputation techniques have been developed to address this issue, they often obsess unnecessary details and may introduce additional noise when making clinical predictions. To tackle this problem, we propose SMART, a Self-Supervised Missing-Aware RepresenTation Learning approach for patient health status prediction, which encodes missing information via elaborated attentions and learns to impute missing values through a novel self-supervised pre-training approach that reconstructs missing data representations in the latent space. By adopting missing-aware attentions and focusing on learning higher-order representations, SMART promotes better generalization and robustness to missing data. We validate the effectiveness of SMART through extensive experiments on six EHR tasks, demonstrating its superiority over state-of-the-art methods.