# To Predict or Not To Predict? Proportionally Masked Autoencoders for Tabular Data Imputation

arXiv: https://arxiv.org/abs/2412.19152 · HTML: https://mlsanity.com/paper/2412.19152/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 34 references that are in the map.
- First posted on arXiv 26 Dec 2024.

## Builds on (most cited first, 5 of 5)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training](https://mlsanity.com/paper/2106.01342/) (2021)
- [Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space](https://mlsanity.com/paper/2310.09656/) (2023)
- [StableDR: Stabilized Doubly Robust Learning for Recommendation on Data Missing Not at Random](https://mlsanity.com/paper/2205.04701/) (2022)

## Built on by (most cited first, 5 of 5)

- [Hybrid Autoencoders for Tabular Data: Leveraging Model-Based Augmentation in Low-Label Settings](https://mlsanity.com/paper/2511.06961/) (2025)
- [MIRRAMS: Learning Robust Tabular Models under Unseen Missingness Shifts](https://mlsanity.com/paper/2507.08280/) (2025)
- [Towards Synthesizing High-Dimensional Tabular Data with Limited Samples](https://mlsanity.com/paper/2503.06444/) (2025)
- [Deep Incomplete Multi-View Clustering via Hierarchical Imputation and Alignment](https://mlsanity.com/paper/2601.09051/) (2026)
- [Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding](https://mlsanity.com/paper/2602.15159/) (2026)

## Abstract (arXiv)

Masked autoencoders (MAEs) have recently demonstrated effectiveness in tabular data imputation. However, due to the inherent heterogeneity of tabular data, the uniform random masking strategy commonly used in MAEs can disrupt the distribution of missingness, leading to suboptimal performance. To address this, we propose a proportional masking strategy for MAEs. Specifically, we first compute the statistics of missingness based on the observed proportions in the dataset, and then generate masks that align with these statistics, ensuring that the distribution of missingness is preserved after masking. Furthermore, we argue that simple MLP-based token mixing offers competitive or often superior performance compared to attention mechanisms while being more computationally efficient, especially in the tabular domain with the inherent heterogeneity. Experimental results validate the effectiveness of the proposed proportional masking strategy across various missing data patterns in tabular datasets. Code is available at: urlhttps://github.com/normal-kim/PMAE.