mlsanityData to 7 Oct 2026

Correlation and variable importance in random forests

arXiv 1310.5726 ↗First posted 21 Oct 2013stat.MEcited by 41

Not on the map. Why?

Abstract (arXiv)

This paper is about variable selection with the random forests algorithm in presence of correlated predictors. In high-dimensional regression or classification frameworks, variable selection is a difficult task, that becomes even more challenging in the presence of highly correlated predictors. Firstly we provide a theoretical study of the permutation importance measure for an additive regression model. This allows us to describe how the correlation between predictors impacts the permutation importance. Our results motivate the use of the Recursive Feature Elimination (RFE) algorithm for variable selection in this context. This algorithm recursively eliminates the variables using permutation importance measure as a ranking criterion. Next various simulation experiments illustrate the efficiency of the RFE algorithm for selecting a small number of variables together with a good prediction error. Finally, this selection algorithm is tested on the Landsat Satellite data from the UCI Machine Learning Repository.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Oct 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 9; 2023: 6; 2024: 9; 2025: 4; 2026: 2.

41 papers.

Sort:

  1. 2021Towards a Science of Human-AI Decision Making: A Survey of Empirical Studiescited by 85
  2. 2021Machine learning pipeline for battery state of health estimationcited by 28
  3. 2022Comparing Explanation Methods for Traditional Machine Learning Models Part 1: An Overview of Current Methods and Quantifying Their Disagreementcited by 14
  4. 2021MDA for random forests: inconsistency, and a practical solution via the Sobol-MDAcited by 10
  5. 2021LEAF: Navigating Concept Drift in Cellular Networkscited by 9
  6. 2021BA-Net: Bridge Attention for Deep Convolutional Neural Networkscited by 9
  7. 2024Feature Inference Attack on Shapley Valuescited by 9
  8. 2022Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streamscited by 8
  9. 2021Feature Importance Explanations for Temporal Black-Box Modelscited by 7
  10. 2021Feature Importance in Gradient Boosting Trees with Cross-Validation Feature Selectioncited by 6
  11. 2022A Generic Shared Attention Mechanism for Various Backbone Neural Networkscited by 5
  12. 2023Exploring the cloud of feature interaction scores in a Rashomon setcited by 5
29 more
  1. 2022Quantile-constrained Wasserstein projections for robust interpretability of numerical and machine learning modelscited by 4
  2. 2023FedScore: A privacy-preserving framework for federated scoring system developmentcited by 4
  3. 2024Time Series Feature Redundancy Paradox: An Empirical Study Based on Mortgage Default Predictioncited by 4
  4. 2023A model-free feature selection technique of feature screening and random forest based recursive feature eliminationcited by 3
  5. 2024Feature Selection for Network Intrusion Detectioncited by 3
  6. 2022Wind power predictions from nowcasts to 4-hour forecasts: a learning approach with variable selectioncited by 2
  7. 2022Comparing Explanation Methods for Traditional Machine Learning Models Part 2: Quantifying Model Explainability Faithfulness and Improvements with Dimensionality Reductioncited by 2
  8. 2023Can predictive models be used for causal inference?cited by 2
  9. 2023MMD-based Variable Importance for Distributional Random Forestcited by 2
  10. 2021A Subspace-based Approach for Dimensionality Reduction and Important Variable Selectioncited by 1
  11. 2022FACT: High-Dimensional Random Forests Inferencecited by 1
  12. 2024A Pipeline for Data-Driven Learning of Topological Features with Applications to Protein Stability Predictioncited by 1
  13. 2025Stability Selection via Variable Decorrelationcited by 1
  14. 2025Decorrelated feature importance from local sample weightingcited by 1
  15. 2021Layer-based Composite Reputation Bootstrappingcited by 0
  16. 2021SalienTrack: providing salient information for semi-automated self-tracking feedback with model explanationscited by 0
  17. 2021Interpretable Models via Pairwise permutations algorithmcited by 0
  18. 2022A machine learning approach to predict the structural and magnetic properties of Heusler alloy familiescited by 0
  19. 2022A Generalized Variable Importance Metric and Estimator for Black Box Machine Learning Modelscited by 0
  20. 2023Random Forests for time-fixed and time-dependent predictors: The DynForest R packagecited by 0
  21. 2024Machine Learning Applications in Traumatic Brain Injury: A Spotlight on Mild TBIcited by 0
  22. 2024Challenges in Variable Importance Ranking Under Correlationcited by 0
  23. 2024EUR-USD Exchange Rate Forecasting Based on Information Fusion with Large Language Models and Deep Learning Methodscited by 0
  24. 2024Two-Stage Human Verification using HandCAPTCHA and Anti-Spoofed Finger Biometrics with Feature Selectioncited by 0
  25. 2024A Central Limit Theorem for the permutation importance measurecited by 0
  26. 2025TaylorPODA: A Taylor Expansion-Based Method to Improve Post-Hoc Attributions for Opaque Modelscited by 0
  27. 2025Mathematical Theory of Collinearity Effects on Machine Learning Variable Importance Measurescited by 0
  28. 2026Pseudo-Feature Padding: A Lightweight Defense Against False Data Injection in Power Gridscited by 0
  29. 2026Benchmarking ConvLSTM for One-Day-Ahead IMDAA Rainfall-Field Prediction across Four Indian Citiescited by 0