# On the consistency of supervised learning with missing values

arXiv: https://arxiv.org/abs/1902.06931 · HTML: https://mlsanity.com/paper/1902.06931/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2019.

## Built on by (most cited first, 30 of 53)

- [Conformal Prediction with Missing Values](https://mlsanity.com/paper/2306.02732/) (2023)
- [Fairness without Imputation: A Decision Tree Approach for Fair Prediction with Missing Values](https://mlsanity.com/paper/2109.10431/) (2021)
- [Benchmarking missing-values approaches for predictive models on health databases](https://mlsanity.com/paper/2202.10580/) (2022)
- [The Missing Indicator Method: From Low to High Dimensions](https://mlsanity.com/paper/2211.09259/) (2022)
- [RedCore: Relative Advantage Aware Cross-modal Representation Learning for Missing Modalities with Imbalanced Missing Rates](https://mlsanity.com/paper/2312.10386/) (2023)
- [Imputation for prediction: beware of diminishing returns](https://mlsanity.com/paper/2407.19804/) (2024)
- [Random features models: a way to study the success of naive imputation](https://mlsanity.com/paper/2402.03839/) (2024)
- [Adapting Fairness Interventions to Missing Values](https://mlsanity.com/paper/2305.19429/) (2023)
- [Retrieve, Merge, Predict: Augmenting Tables with Data Lakes](https://mlsanity.com/paper/2402.06282/) (2024)
- [High-dimensional logistic regression with missing data: Imputation, regularization, and universality](https://mlsanity.com/paper/2410.01093/) (2024)
- [Knockout: A simple way to handle missing inputs](https://mlsanity.com/paper/2405.20448/) (2024)
- [Modeling Hierarchical Spaces: A Review and Unified Framework for Surrogate-Based Architecture Design](https://mlsanity.com/paper/2506.22621/) (2025)
- [MINTY: Rule-based Models that Minimize the Need for Imputing Features with Missing Values](https://mlsanity.com/paper/2311.14108/) (2023)
- [Learning Data-Driven Uncertainty Set Partitions for Robust and Adaptive Energy Forecasting with Missing Data](https://mlsanity.com/paper/2503.20410/) (2025)
- [Deep learning with missing data](https://mlsanity.com/paper/2504.15388/) (2025)
- [Model-based Clustering with Missing Not At Random Data](https://mlsanity.com/paper/2112.10425/) (2021)
- [RIGID: Robust Linear Regression with Missing Data](https://mlsanity.com/paper/2205.13635/) (2022)
- [No imputation without representation](https://mlsanity.com/paper/2206.14254/) (2022)
- [Density Ratio Estimation and Neyman Pearson Classification with Missing Data](https://mlsanity.com/paper/2302.10655/) (2023)
- [Algorithmic Recourse with Missing Values](https://mlsanity.com/paper/2304.14606/) (2023)
- [Adaptive Optimization for Prediction with Missing Data](https://mlsanity.com/paper/2402.01543/) (2024)
- [Robust prediction under missingness shifts](https://mlsanity.com/paper/2406.16484/) (2024)
- [missForestPredict -- Missing data imputation for prediction settings](https://mlsanity.com/paper/2407.03379/) (2024)
- [Federated Causal Inference from Multi-Site Observational Data via Propensity Score Aggregation](https://mlsanity.com/paper/2505.17961/) (2025)
- [Missing Data in Signal Processing and Machine Learning: Models, Methods and Modern Approaches](https://mlsanity.com/paper/2506.01696/) (2025)
- [Accelerated Learning on Large Scale Screens using Generative Library Models](https://mlsanity.com/paper/2510.16612/) (2025)
- [Minimax rate of consistency for linear models with missing values](https://mlsanity.com/paper/2202.01463/) (2022)
- [PROMISSING: Pruning Missing Values in Neural Networks](https://mlsanity.com/paper/2206.01640/) (2022)
- [Imputation of missing values in multi-view data](https://mlsanity.com/paper/2210.14484/) (2022)
- [Data Imputation by Pursuing Better Classification: A Supervised Kernel-Based Method](https://mlsanity.com/paper/2405.07800/) (2024)

## Abstract (arXiv)

In many application settings, the data have missing entries which make analysis challenging. An abundant literature addresses missing values in an inferential framework: estimating parameters and their variance from incomplete tables. Here, we consider supervised-learning settings: predicting a target when missing values appear in both training and testing data. We show the consistency of two approaches in prediction. A striking result is that the widely-used method of imputing with a constant, such as the mean prior to learning is consistent when missing values are not informative. This contrasts with inferential settings where mean imputation is pointed at for distorting the distribution of the data. That such a simple approach can be consistent is important in practice. We also show that a predictor suited for complete observations can predict optimally on incomplete data, through multiple imputation. Finally, to compare imputation with learning directly with a model that accounts for missing values, we analyze further decision trees. These can naturally tackle empirical risk minimization with missing values, due to their ability to handle the half-discrete nature of incomplete variables. After comparing theoretically and empirically different missing values strategies in trees, we recommend using the "missing incorporated in attribute" method as it can handle both non-informative and informative missing values.