# ReMasker: Imputing Tabular Data with Masked Autoencoding

arXiv: https://arxiv.org/abs/2309.13793 · HTML: https://mlsanity.com/paper/2309.13793/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 25 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [On Embeddings for Numerical Features in Tabular Deep Learning](https://mlsanity.com/paper/2203.05556/) (2022)
- [TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up](https://mlsanity.com/paper/2102.07074/) (2021)
- [StyleSwin: Transformer-based GAN for High-resolution Image Generation](https://mlsanity.com/paper/2112.10762/) (2021)
- [Handling Incomplete Heterogeneous Data using VAEs](https://mlsanity.com/paper/1807.03653/) (2018)
- [Generative Adversarial Transformers](https://mlsanity.com/paper/2103.01209/) (2021)
- [Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares](https://mlsanity.com/paper/1410.2596/) (2014)
- [Missing Data Imputation using Optimal Transport](https://mlsanity.com/paper/2002.03860/) (2020)
- [HyperImpute: Generalized Iterative Imputation with Automatic Model Selection](https://mlsanity.com/paper/2206.07769/) (2022)
- [Towards Deeper Understanding of Variational Autoencoding Models](https://mlsanity.com/paper/1702.08658/) (2017)
- [MIRACLE: Causally-Aware Imputation via Learning Missing Data Mechanisms](https://mlsanity.com/paper/2111.03187/) (2021)
- [Understanding Masked Image Modeling via Learning Occlusion Invariant Feature](https://mlsanity.com/paper/2208.04164/) (2022)
- [Identifiable Generative Models for Missing Not at Random Data Imputation](https://mlsanity.com/paper/2110.14708/) (2021)
- [MIWAE: Deep Generative Modelling and Imputation of Incomplete Data](https://mlsanity.com/paper/1812.02633/) (2018)
- [MisConv: Convolutional Neural Networks for Missing Data](https://mlsanity.com/paper/2110.14010/) (2021)

## Built on by (most cited first, 30 of 37)

- [LSM-2: Learning from Incomplete Wearable Sensor Data](https://mlsanity.com/paper/2506.05321/) (2025)
- [DiffPuter: Empowering Diffusion Models for Missing Data Imputation](https://mlsanity.com/paper/2405.20690/) (2024)
- [On LLM-Enhanced Mixed-Type Data Imputation with High-Order Message Passing](https://mlsanity.com/paper/2501.02191/) (2025)
- [TabSeq: A Framework for Deep Learning on Tabular Data via Sequential Ordering](https://mlsanity.com/paper/2410.13203/) (2024)
- [Diffusion-nested Auto-Regressive Synthesis of Heterogeneous Tabular Data](https://mlsanity.com/paper/2410.21523/) (2024)
- [CACTI: Leveraging Copy Masking and Contextual Information to Improve Tabular Data Imputation](https://mlsanity.com/paper/2506.02306/) (2025)
- [One Pipeline, Many Transformers: Pattern-Specific Imputation Specialists for Tabular Missing Data](https://mlsanity.com/paper/2510.02625/) (2025)
- [MaskSDM with Shapley values to improve flexibility, robustness, and explainability in species distribution modeling](https://mlsanity.com/paper/2503.13057/) (2025)
- [Representation Learning of Lab Values via Masked AutoEncoders](https://mlsanity.com/paper/2501.02648/) (2025)
- [Leveraging Machine Unlearning for Cost-Efficient Preference Alignment](https://mlsanity.com/paper/2504.06659/) (2025)
- [Impute-MACFM: Imputation based on Mask-Aware Flow Matching](https://mlsanity.com/paper/2509.23126/) (2025)
- [Interpretable Generative and Discriminative Learning for Multimodal and Incomplete Clinical Data](https://mlsanity.com/paper/2510.09513/) (2025)
- [Masked Language Modeling Becomes Conditional Density Estimation for Tabular Data Synthesis](https://mlsanity.com/paper/2405.20602/) (2024)
- [TAEGAN: Generating Synthetic Tabular Data For Data Augmentation](https://mlsanity.com/paper/2410.01933/) (2024)
- [Imputation-free Learning of Tabular Data with Missing Values using Incremental Feature Partitions in Transformer](https://mlsanity.com/paper/2504.14610/) (2025)
- [DepMicroDiff: Diffusion-Based Dependency-Aware Multimodal Imputation for Microbiome Data](https://mlsanity.com/paper/2507.23676/) (2025)
- [M3-Impute: Mask-guided Representation Learning for Missing Value Imputation](https://mlsanity.com/paper/2410.08794/) (2024)
- [IMAGIC-500: IMputation benchmark on A Generative Imaginary Country (500k samples)](https://mlsanity.com/paper/2506.08844/) (2025)
- [Meta-Imputation Balanced (MIB): An Ensemble Approach for Handling Missing Data in Biomedical Machine Learning](https://mlsanity.com/paper/2509.03316/) (2025)
- [An Interdisciplinary and Cross-Task Review on Missing Data Imputation](https://mlsanity.com/paper/2511.01196/) (2025)
- [Masked Autoencoder Joint Learning for Robust Spitzoid Tumor Classification](https://mlsanity.com/paper/2511.19535/) (2025)
- [Weighted Conformal Prediction Provides Adaptive and Valid Mask-Conditional Coverage for General Missing Data Mechanisms](https://mlsanity.com/paper/2512.14221/) (2025)
- [Nearly Optimal Bayesian Inference for Structural Missingness](https://mlsanity.com/paper/2601.18500/) (2026)
- [Harpoon: Generalised Manifold Guidance for Conditional Tabular Diffusion](https://mlsanity.com/paper/2602.07875/) (2026)
- [Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding](https://mlsanity.com/paper/2602.15159/) (2026)
- [Demystifying the Optimal Fair Classifier in Multi-Class Classification](https://mlsanity.com/paper/2606.00656/) (2026)
- [Learning What Not to Impute: An Uncertainty-Aware Diffusion Framework for Meaningful Missingness](https://mlsanity.com/paper/2606.05073/) (2026)
- [TabSODA: Tabular Diffusion based Imputation with Skip Pattern Detection and Ordinal Awareness](https://mlsanity.com/paper/2606.05361/) (2026)
- [SafeImpute: Reliable Clinical Data Imputation via Conformal Selection](https://mlsanity.com/paper/2607.05613/) (2026)
- [Imputation Meets Clustering: Exploiting Latent Subgroup Structure for Missing Data Recovery](https://mlsanity.com/paper/2607.06930/) (2026)

## Abstract (arXiv)

We present ReMasker, a new method of imputing missing values in tabular data by extending the masked autoencoding framework. Compared with prior work, ReMasker is both simple -- besides the missing values (i.e., naturally masked), we randomly ``re-mask'' another set of values, optimize the autoencoder by reconstructing this re-masked set, and apply the trained model to predict the missing values; and effective -- with extensive evaluation on benchmark datasets, we show that ReMasker performs on par with or outperforms state-of-the-art methods in terms of both imputation fidelity and utility under various missingness settings, while its performance advantage often increases with the ratio of missing data. We further explore theoretical justification for its effectiveness, showing that ReMasker tends to learn missingness-invariant representations of tabular data. Our findings indicate that masked modeling represents a promising direction for further research on tabular data imputation. The code is publicly available.