# Imputation for prediction: beware of diminishing returns

arXiv: https://arxiv.org/abs/2407.19804 · HTML: https://mlsanity.com/paper/2407.19804/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training](https://mlsanity.com/paper/2106.01342/) (2021)
- [Missing Data Imputation using Optimal Transport](https://mlsanity.com/paper/2002.03860/) (2020)
- [HyperImpute: Generalized Iterative Imputation with Automatic Model Selection](https://mlsanity.com/paper/2206.07769/) (2022)
- [What's a good imputation to predict with missing values?](https://mlsanity.com/paper/2106.00311/) (2021)
- [On the consistency of supervised learning with missing values](https://mlsanity.com/paper/1902.06931/) (2019)
- [Benchmarking missing-values approaches for predictive models on health databases](https://mlsanity.com/paper/2202.10580/) (2022)
- [Classification of datasets with imputed missing values: does imputation quality matter?](https://mlsanity.com/paper/2206.08478/) (2022)
- [The Missing Indicator Method: From Low to High Dimensions](https://mlsanity.com/paper/2211.09259/) (2022)
- [Does imputation matter? Benchmark for predictive models](https://mlsanity.com/paper/2007.02837/) (2020)
- [Random features models: a way to study the success of naive imputation](https://mlsanity.com/paper/2402.03839/) (2024)
- [On the Relation between Prediction and Imputation Accuracy under Missing Covariates](https://mlsanity.com/paper/2112.05248/) (2021)
- [Missing Data Imputation for Supervised Learning](https://mlsanity.com/paper/1610.09075/) (2016)

## Built on by (most cited first, 13 of 13)

- [Missing Data in Signal Processing and Machine Learning: Models, Methods and Modern Approaches](https://mlsanity.com/paper/2506.01696/) (2025)
- [TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks](https://mlsanity.com/paper/2606.02384/) (2026)
- [Imputation-free Learning of Tabular Data with Missing Values using Incremental Feature Partitions in Transformer](https://mlsanity.com/paper/2504.14610/) (2025)
- [Missing Data Multiple Imputation for Tabular Q-Learning in Online RL](https://mlsanity.com/paper/2510.10709/) (2025)
- [A Multimodal Deep Learning Framework for Predicting ICU Deterioration: Integrating ECG Waveforms with Clinical Data and Clinician Benchmarking](https://mlsanity.com/paper/2601.06645/) (2026)
- [Optimal Transport with Heterogeneously Missing Data](https://mlsanity.com/paper/2505.17291/) (2025)
- [On the Inherent Privacy Amplification of Missing Data](https://mlsanity.com/paper/2602.01928/) (2026)
- [Handling Covariate Mismatch in Collaborative Linear Prediction](https://mlsanity.com/paper/2602.02083/) (2026)
- [Characterizing the Predictive Impact of Modalities with Supervised Latent-Variable Modeling](https://mlsanity.com/paper/2602.16979/) (2026)
- [Hypernetworks for Dynamic Feature Selection](https://mlsanity.com/paper/2605.12278/) (2026)
- [Distributionally Faithful Imputation via Positive Semi-Definite Kernel Density Estimation](https://mlsanity.com/paper/2607.07767/) (2026)
- [Hybrid Methods for Robust Tabular Data Imputation](https://mlsanity.com/paper/2609.39613/) (2026)
- [Integrated Imputation-Classification for Supervised Learning with Missing Data](https://mlsanity.com/paper/2610.04273/) (2026)

## Abstract (arXiv)

Missing values are prevalent across various fields, posing challenges for training and deploying predictive models. In this context, imputation is a common practice, driven by the hope that accurate imputations will enhance predictions. However, recent theoretical and empirical studies indicate that simple constant imputation can be consistent and competitive. This empirical study aims at clarifying if and when investing in advanced imputation methods yields significantly better predictions. Relating imputation and predictive accuracies across combinations of imputation and predictive models on 19 datasets, we show that imputation accuracy matters less i) when using expressive models, ii) when incorporating missingness indicators as complementary inputs, iii) matters much more for generated linear outcomes than for real-data outcomes. Interestingly, we also show that the use of the missingness indicator is beneficial to the prediction performance, even in MCAR scenarios. Overall, on real-data with powerful models, improving imputation only has a minor effect on prediction performance. Thus, investing in better imputations for improved predictions often offers limited benefits.