# Variable selection with missing data in both covariates and outcomes: Imputation and machine learning

arXiv: https://arxiv.org/abs/2104.02769 · HTML: https://mlsanity.com/paper/2104.02769/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 66 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Random Forest Missing Data Algorithms](https://mlsanity.com/paper/1701.05305/) (2017)
- [Estimation of Causal Effects of Multiple Treatments in Observational Studies with a Binary Outcome](https://mlsanity.com/paper/2001.06483/) (2020)
- [Estimating heterogeneous survival treatment effect in observational data using machine learning](https://mlsanity.com/paper/2008.07044/) (2020)
- [Prediction with Missing Data via Bayesian Additive Regression Trees](https://mlsanity.com/paper/1306.0618/) (2013)
- [Variable selection for BART: An application to gene regulation](https://mlsanity.com/paper/1310.4887/) (2013)
- [Causal comparative effectiveness analysis of dynamic continuous-time treatment initiation rules with sparsely measured outcomes and death](https://mlsanity.com/paper/1904.01668/) (2019)
- [Estimation of causal effects of multiple treatments in healthcare database studies with rare outcomes](https://mlsanity.com/paper/2008.07687/) (2020)
- [A flexible sensitivity analysis approach for unmeasured confounding with multiple treatments and a binary outcome with application to SEER-Medicare lung cancer data](https://mlsanity.com/paper/2012.06093/) (2020)
- [Modeling the Causal Effect of Treatment Initiation Time on Survival: Application to HIV/TB Co-infection](https://mlsanity.com/paper/1904.01932/) (2019)

## Abstract (arXiv)

The missing data issue is ubiquitous in health studies. Variable selection in the presence of both missing covariates and outcomes is an important statistical research topic but has been less studied. Existing literature focuses on parametric regression techniques that provide direct parameter estimates of the regression model. Flexible nonparametric machine learning methods considerably mitigate the reliance on the parametric assumptions, but do not provide as naturally defined variable importance measure as the covariate effect native to parametric models. We investigate a general variable selection approach when both the covariates and outcomes can be missing at random and have general missing data patterns. This approach exploits the flexibility of machine learning modeling techniques and bootstrap imputation, which is amenable to nonparametric methods in which the covariate effects are not directly available. We conduct expansive simulations investigating the practical operating characteristics of the proposed variable selection approach, when combined with four tree-based machine learning methods, XGBoost, Random Forests, Bayesian Additive Regression Trees (BART) and Conditional Random Forests, and two commonly used parametric methods, lasso and backward stepwise selection. Numeric results suggest that when combined with bootstrap imputation, XGBoost and BART have the overall best variable selection performance with respect to the F1 score and Type I error across various settings. In general, there is no significant difference in the variable selection performance due to imputation methods. We further demonstrate the methods via a case study of risk factors for 3-year incidence of metabolic syndrome with data from the Study of Women's Health Across the Nation.