# Not Another Imputation Method: A Transformer-based Model for Missing Values in Tabular Datasets

arXiv: https://arxiv.org/abs/2407.11540 · HTML: https://mlsanity.com/paper/2407.11540/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [Handling Missing Data with Graph Representation Learning](https://mlsanity.com/paper/2010.16418/) (2020)
- [Analyzing Transformers in Embedding Space](https://mlsanity.com/paper/2209.02535/) (2022)
- [Handling missing values in healthcare data: A systematic review of deep learning-based imputation techniques](https://mlsanity.com/paper/2210.08258/) (2022)
- [Benchmarking missing-values approaches for predictive models on health databases](https://mlsanity.com/paper/2202.10580/) (2022)
- [A Deep Learning Approach for Overall Survival Prediction in Lung Cancer with Missing Values](https://mlsanity.com/paper/2307.11465/) (2023)

## Built on by (most cited first, 10 of 10)

- [MARIA: a Multimodal Transformer Model for Incomplete Healthcare Data](https://mlsanity.com/paper/2412.14810/) (2024)
- [Beyond a Single Mode: GAN Ensembles for Diverse Medical Data Generation](https://mlsanity.com/paper/2503.24258/) (2025)
- [Imputation-free Learning of Tabular Data with Missing Values using Incremental Feature Partitions in Transformer](https://mlsanity.com/paper/2504.14610/) (2025)
- [Handling Missing Modalities in Multimodal Survival Prediction for Non-Small Cell Lung Cancer](https://mlsanity.com/paper/2601.10386/) (2026)
- [Learning Causal Orderings for In-Context Tabular Prediction](https://mlsanity.com/paper/2605.22335/) (2026)
- [SHIFT: Survival Prediction from Incomplete and Heterogeneous Genomic Data](https://mlsanity.com/paper/2607.07725/) (2026)
- [Learning from Limited and Incomplete Data: A Multimodal Framework for Predicting Pathological Response in NSCLC](https://mlsanity.com/paper/2603.15100/) (2026)
- [Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts](https://mlsanity.com/paper/2607.11656/) (2026)
- [From Attention Masks to Inert Zero-Vector Tokens: OAttention and O-Closure for Token Dynamics](https://mlsanity.com/paper/2608.21174/) (2026)
- [Marformer: A Transformer for Predicting Missing Data Distributions](https://mlsanity.com/paper/2610.12379/) (2026)

## Abstract (arXiv)

Handling missing values in tabular datasets presents a significant challenge in training and testing artificial intelligence models, an issue usually addressed using imputation techniques. Here we introduce "Not Another Imputation Method" (NAIM), a novel transformer-based model specifically designed to address this issue without the need for traditional imputation techniques. NAIM's ability to avoid the necessity of imputing missing values and to effectively learn from available data relies on two main techniques: the use of feature-specific embeddings to encode both categorical and numerical features also handling missing inputs; the modification of the masked self-attention mechanism to completely mask out the contributions of missing data. Additionally, a novel regularization technique is introduced to enhance the model's generalization capability from incomplete data. We extensively evaluated NAIM on 5 publicly available tabular datasets, demonstrating its superior performance over 6 state-of-the-art machine learning models and 5 deep learning models, each paired with 3 different imputation techniques when necessary. The results highlight the efficacy of NAIM in improving predictive performance and resilience in the presence of missing data. To facilitate further research and practical application in handling missing data without traditional imputation methods, we made the code for NAIM available at https://github.com/cosbidev/NAIM.