# Random features models: a way to study the success of naive imputation

arXiv: https://arxiv.org/abs/2402.03839 · HTML: https://mlsanity.com/paper/2402.03839/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 13 of 13)

- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Non-strongly-convex smooth stochastic approximation with convergence rate O(1/n)](https://mlsanity.com/paper/1306.2119/) (2013)
- [Early stopping and non-parametric regression: An optimal data-dependent stopping rule](https://mlsanity.com/paper/1306.3574/) (2013)
- [Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression](https://mlsanity.com/paper/1602.05419/) (2016)
- [What's a good imputation to predict with missing values?](https://mlsanity.com/paper/2106.00311/) (2021)
- [On the consistency of supervised learning with missing values](https://mlsanity.com/paper/1902.06931/) (2019)
- [Learning with SGD and Random Features](https://mlsanity.com/paper/1807.06343/) (2018)
- [On Robustness of Principal Component Regression](https://mlsanity.com/paper/1902.10920/) (2019)
- [Benchmarking missing-values approaches for predictive models on health databases](https://mlsanity.com/paper/2202.10580/) (2022)
- [Linear predictor on linearly-generated data with missing values: non consistency and solutions](https://mlsanity.com/paper/2002.00658/) (2020)
- [An elementary analysis of ridge regression with random design](https://mlsanity.com/paper/2203.08564/) (2022)

## Built on by (most cited first, 7 of 7)

- [Imputation for prediction: beware of diminishing returns](https://mlsanity.com/paper/2407.19804/) (2024)
- [Random Feature Representation Boosting](https://mlsanity.com/paper/2501.18283/) (2025)
- [Missing Data in Signal Processing and Machine Learning: Models, Methods and Modern Approaches](https://mlsanity.com/paper/2506.01696/) (2025)
- [Model Evaluation in the Dark: Robust Classifier Metrics with Missing Labels](https://mlsanity.com/paper/2504.18385/) (2025)
- [Optimal Transport with Heterogeneously Missing Data](https://mlsanity.com/paper/2505.17291/) (2025)
- [Handling Covariate Mismatch in Collaborative Linear Prediction](https://mlsanity.com/paper/2602.02083/) (2026)
- [Increasing Missingness to Reduce Bias: Richardson-SGD with Missing Data](https://mlsanity.com/paper/2605.19641/) (2026)

## Abstract (arXiv)

Constant (naive) imputation is still widely used in practice as this is a first easy-to-use technique to deal with missing data. Yet, this simple method could be expected to induce a large bias for prediction purposes, as the imputed input may strongly differ from the true underlying data. However, recent works suggest that this bias is low in the context of high-dimensional linear predictors when data is supposed to be missing completely at random (MCAR). This paper completes the picture for linear predictors by confirming the intuition that the bias is negligible and that surprisingly naive imputation also remains relevant in very low dimension.To this aim, we consider a unique underlying random features model, which offers a rigorous framework for studying predictive performances, whilst the dimension of the observed features varies.Building on these theoretical results, we establish finite-sample bounds on stochastic gradient (SGD) predictors applied to zero-imputed data, a strategy particularly well suited for large-scale learning.If the MCAR assumption appears to be strong, we show that similar favorable behaviors occur for more complex missing data scenarios.