# Which Imputation Fits Which Feature Selection Method? A Survey-Based Simulation Study

arXiv: https://arxiv.org/abs/2412.13570 · HTML: https://mlsanity.com/paper/2412.13570/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 83 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Builds on (most cited first, 11 of 11)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R](https://mlsanity.com/paper/1508.04409/) (2015)
- [Hyperparameters and Tuning Strategies for Random Forest](https://mlsanity.com/paper/1804.03515/) (2018)
- [Applied Causal Inference Powered by ML and AI](https://mlsanity.com/paper/2403.02467/) (2024)
- [A Guide to Feature Importance Methods for Scientific Inference](https://mlsanity.com/paper/2404.12862/) (2024)
- [On the role of benchmarking data sets and simulations in method comparison studies](https://mlsanity.com/paper/2208.01457/) (2022)
- [On the Relation between Prediction and Imputation Accuracy under Missing Covariates](https://mlsanity.com/paper/2112.05248/) (2021)
- [Multiple Imputation Through XGBoost](https://mlsanity.com/paper/2106.01574/) (2021)
- [Evaluating tree-based imputation methods as an alternative to MICE PMM for drawing inference in empirical studies](https://mlsanity.com/paper/2401.09602/) (2024)
- [Imputation procedures in surveys using nonparametric and machine learning methods: an empirical comparison](https://mlsanity.com/paper/2007.06298/) (2020)
- [Adapting tree-based multiple imputation methods for multi-level data? A simulation study](https://mlsanity.com/paper/2401.14161/) (2024)

## Built on by (most cited first, 2 of 2)

- [Evaluating Counterfactual Explanation Methods on Incomplete Inputs](https://mlsanity.com/paper/2604.08004/) (2026)
- [Cross-Course Generalizability of SRL-Aligned Predictive Models Using Digital Learning Traces](https://mlsanity.com/paper/2604.22812/) (2026)

## Abstract (arXiv)

Tree-based learning methods such as Random Forest and XGBoost are still the gold-standard prediction methods for tabular data. Feature importance measures are usually considered for feature selection as well as to assess the effect of features on the outcome variables in the model. This also applies to survey data, which are frequently encountered in the social sciences and official statistics. These types of datasets often present the challenge of missing values. The typical solution is to impute the missing data before applying the learning method. However, given the large number of possible imputation methods available, the question arises as to which should be chosen to achieve the 'best' reflection of feature importance and feature selection in subsequent analyses. In the present paper, we investigate this question in a survey-based simulation study for eight state-of-the art imputation methods and three learners. The imputation methods comprise listwise deletion, three MICE options, four textttmissRanger options as well as the recently proposed mixGBoost imputation approach. As learners, we consider the two most common tree-based methods, Random Forest and XGBoost, and an interpretable linear model with regularization.