# What's a good imputation to predict with missing values?

arXiv: https://arxiv.org/abs/2106.00311 · HTML: https://mlsanity.com/paper/2106.00311/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 23 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 4 of 4)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [Linear predictor on linearly-generated data with missing values: non consistency and solutions](https://mlsanity.com/paper/2002.00658/) (2020)

## Built on by (most cited first, 30 of 62)

- [HyperImpute: Generalized Iterative Imputation with Automatic Model Selection](https://mlsanity.com/paper/2206.07769/) (2022)
- [Conformal Prediction with Missing Values](https://mlsanity.com/paper/2306.02732/) (2023)
- [Benchmarking missing-values approaches for predictive models on health databases](https://mlsanity.com/paper/2202.10580/) (2022)
- [The Missing Indicator Method: From Low to High Dimensions](https://mlsanity.com/paper/2211.09259/) (2022)
- [Domain Adaptation under Missingness Shift](https://mlsanity.com/paper/2211.02093/) (2022)
- [Exploring and Exploiting Uncertainty for Incomplete Multi-View Classification](https://mlsanity.com/paper/2304.05165/) (2023)
- [Imputation for prediction: beware of diminishing returns](https://mlsanity.com/paper/2407.19804/) (2024)
- [TimeCHEAT: A Channel Harmony Strategy for Irregularly Sampled Multivariate Time Series Analysis](https://mlsanity.com/paper/2412.12886/) (2024)
- [Evaluation of Active Feature Acquisition Methods for Time-varying Feature Settings](https://mlsanity.com/paper/2312.01530/) (2023)
- [Sharing pattern submodels for prediction with missing values](https://mlsanity.com/paper/2206.11161/) (2022)
- [Random features models: a way to study the success of naive imputation](https://mlsanity.com/paper/2402.03839/) (2024)
- [Interpretable Generalized Additive Models for Datasets with Missing Values](https://mlsanity.com/paper/2412.02646/) (2024)
- [Multivariate Standardized Residuals for Conformal Prediction](https://mlsanity.com/paper/2507.20941/) (2025)
- [Adapting Fairness Interventions to Missing Values](https://mlsanity.com/paper/2305.19429/) (2023)
- [Evaluation of Active Feature Acquisition Methods for Static Feature Settings](https://mlsanity.com/paper/2312.03619/) (2023)
- [Learnable Prompt as Pseudo-Imputation: Rethinking the Necessity of Traditional EHR Data Imputation in Downstream Clinical Prediction](https://mlsanity.com/paper/2401.16796/) (2024)
- [Knockout: A simple way to handle missing inputs](https://mlsanity.com/paper/2405.20448/) (2024)
- [Leveraging Predictive Equivalence in Decision Trees](https://mlsanity.com/paper/2506.14143/) (2025)
- [Numerical Data Imputation for Multimodal Data Sets: A Probabilistic Nearest-Neighbor Kernel Density Approach](https://mlsanity.com/paper/2306.16906/) (2023)
- [MINTY: Rule-based Models that Minimize the Need for Imputing Features with Missing Values](https://mlsanity.com/paper/2311.14108/) (2023)
- [Machine-learning-based particle identification with missing data](https://mlsanity.com/paper/2401.01905/) (2023)
- [Deep learning with missing data](https://mlsanity.com/paper/2504.15388/) (2025)
- [RIGID: Robust Linear Regression with Missing Data](https://mlsanity.com/paper/2205.13635/) (2022)
- [No imputation without representation](https://mlsanity.com/paper/2206.14254/) (2022)
- [Deeply-Learned Generalized Linear Models with Missing Data](https://mlsanity.com/paper/2207.08911/) (2022)
- [Algorithmic Recourse with Missing Values](https://mlsanity.com/paper/2304.14606/) (2023)
- [FedECA: Federated External Control Arms for Causal Inference with Time-To-Event Data in Distributed Settings](https://mlsanity.com/paper/2311.16984/) (2023)
- [Adaptive Optimization for Prediction with Missing Data](https://mlsanity.com/paper/2402.01543/) (2024)
- [Robust prediction under missingness shifts](https://mlsanity.com/paper/2406.16484/) (2024)
- [Impact of Missing Values in Machine Learning: A Comprehensive Analysis](https://mlsanity.com/paper/2410.08295/) (2024)

## Abstract (arXiv)

How to learn a good predictor on data with missing values? Most efforts focus on first imputing as well as possible and second learning on the completed data to predict the outcome. Yet, this widespread practice has no theoretical grounding. Here we show that for almost all imputation functions, an impute-then-regress procedure with a powerful learner is Bayes optimal. This result holds for all missing-values mechanisms, in contrast with the classic statistical results that require missing-at-random settings to use imputation in probabilistic modeling. Moreover, it implies that perfect conditional imputation is not needed for good prediction asymptotically. In fact, we show that on perfectly imputed data the best regression function will generally be discontinuous, which makes it hard to learn. Crafting instead the imputation so as to leave the regression function unchanged simply shifts the problem to learning discontinuous imputations. Rather, we suggest that it is easier to learn imputation and regression jointly. We propose such a procedure, adapting NeuMiss, a neural network capturing the conditional links across observed and unobserved variables whatever the missing-value pattern. Experiments confirm that joint imputation and regression through NeuMiss is better than various two step procedures in our experiments with finite number of samples.