# Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding

arXiv: https://arxiv.org/abs/2602.15159 · HTML: https://mlsanity.com/paper/2602.15159/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 54 references that are in the map.
- First posted on arXiv 16 Feb 2026.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [Set Functions for Time Series](https://mlsanity.com/paper/1909.12064/) (2019)
- [Why Tabular Foundation Models Should Be a Research Priority](https://mlsanity.com/paper/2405.01147/) (2024)
- [Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series](https://mlsanity.com/paper/2107.14293/) (2021)
- [A Survey on Principles, Models and Methods for Learning from Irregularly Sampled Time Series](https://mlsanity.com/paper/2012.00168/) (2020)
- [ReMasker: Imputing Tabular Data with Masked Autoencoding](https://mlsanity.com/paper/2309.13793/) (2023)
- [LSM-2: Learning from Incomplete Wearable Sensor Data](https://mlsanity.com/paper/2506.05321/) (2025)
- [DuETT: Dual Event Time Transformer for Electronic Health Records](https://mlsanity.com/paper/2304.13017/) (2023)
- [An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detection](https://mlsanity.com/paper/1708.05894/) (2017)
- [Event-Based Contrastive Learning for Medical Time Series](https://mlsanity.com/paper/2312.10308/) (2023)
- [Clustering with UMAP: Why and How Connectivity Matters](https://mlsanity.com/paper/2108.05525/) (2021)
- [Labrador: Exploring the Limits of Masked Language Modeling for Laboratory Data](https://mlsanity.com/paper/2312.11502/) (2023)
- [To Predict or Not To Predict? Proportionally Masked Autoencoders for Tabular Data Imputation](https://mlsanity.com/paper/2412.19152/) (2024)
- [Representation Learning of Lab Values via Masked AutoEncoders](https://mlsanity.com/paper/2501.02648/) (2025)
- [From Linearity to Non-Linearity: How Masked Autoencoders Capture Spatial Correlations](https://mlsanity.com/paper/2508.15404/) (2025)

## Abstract (arXiv)

Electronic health records (EHR) arrive masked. Clinicians order measurements selectively, and any patient table thus contains only a subset of the values that characterize the underlying physiological state. Prior masked modeling approaches on EHR data either impute the table before learning, represent missingness through a dedicated placeholder signal, or optimize solely for imputation, which limits the representations they learn for downstream clinical tasks and carries every unobserved entry through the encoder. We introduce AID-MAE, an Augmented-Intrinsic Dual-Masked Autoencoder that learns directly from incomplete tables by combining the intrinsic mask the record already carries with an augmented mask that hides a subset of observed values for reconstruction during pretraining. Neither type of masked entry enters the encoder, so attention operates only over what was observed. AID-MAE achieves consistent improvements over strong baselines across multiple clinical tasks on two datasets. Across experiments, we discuss that recovering the missing entries is not a prerequisite for learning and show that the representations learned carry clinical structure without supervision.