# Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approach

arXiv: https://arxiv.org/abs/2006.04628 · HTML: https://mlsanity.com/paper/2006.04628/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2020.

## Built on by (most cited first, 30 of 37)

- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Explaining Hyperparameter Optimization via Partial Dependence Plots](https://mlsanity.com/paper/2111.04820/) (2021)
- [Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Process](https://mlsanity.com/paper/2109.01433/) (2021)
- [Dear XAI Community, We Need to Talk! Fundamental Misconceptions in Current XAI Research](https://mlsanity.com/paper/2306.04292/) (2023)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [REPID: Regional Effect Plots with implicit Interaction Detection](https://mlsanity.com/paper/2202.07254/) (2022)
- [Grouped Feature Importance and Combined Features Effect Plot](https://mlsanity.com/paper/2104.11688/) (2021)
- [A Guide to Feature Importance Methods for Scientific Inference](https://mlsanity.com/paper/2404.12862/) (2024)
- [Unifying Feature-Based Explanations with Functional ANOVA and Cooperative Game Theory](https://mlsanity.com/paper/2412.17152/) (2024)
- [Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streams](https://mlsanity.com/paper/2209.01939/) (2022)
- [RHALE: Robust and Heterogeneity-aware Accumulated Local Effects](https://mlsanity.com/paper/2309.11193/) (2023)
- [Conditional Feature Importance for Mixed Data](https://mlsanity.com/paper/2210.03047/) (2022)
- [Interpretable machine learning for time-to-event prediction in medicine and healthcare](https://mlsanity.com/paper/2303.09817/) (2023)
- [Postprocessing of Ensemble Weather Forecasts Using Permutation-invariant Neural Networks](https://mlsanity.com/paper/2309.04452/) (2023)
- [Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency](https://mlsanity.com/paper/2111.15156/) (2021)
- [Model-agnostic variable importance for predictive uncertainty: an entropy-based approach](https://mlsanity.com/paper/2310.12842/) (2023)
- [iPDP: On Partial Dependence Plots in Dynamic Modeling Scenarios](https://mlsanity.com/paper/2306.07775/) (2023)
- [iSAGE: An Incremental Version of SAGE for Online Explanation on Data Streams](https://mlsanity.com/paper/2303.01181/) (2023)
- [On the Relationship Between Interpretability and Explainability in Machine Learning](https://mlsanity.com/paper/2311.11491/) (2023)
- [Transforming Feature Space to Interpret Machine Learning Models](https://mlsanity.com/paper/2104.04295/) (2021)
- [Examining spatial heterogeneity of ridesourcing demand determinants with explainable machine learning](https://mlsanity.com/paper/2209.07980/) (2022)
- [Explaining and visualizing black-box models through counterfactual paths](https://mlsanity.com/paper/2307.07764/) (2023)
- [Explainable AI for clinical risk prediction: a survey of concepts, methods, and modalities](https://mlsanity.com/paper/2308.08407/) (2023)
- [Developing Guidelines for Functionally-Grounded Evaluation of Explainable Artificial Intelligence using Tabular Data](https://mlsanity.com/paper/2410.12803/) (2024)
- [CohEx: A Generalized Framework for Cohort Explanation](https://mlsanity.com/paper/2410.13190/) (2024)
- [FINCH: Locally Visualizing Higher-Order Feature Interactions in Black Box Models](https://mlsanity.com/paper/2503.16445/) (2025)
- [Towards interpreting ML-based automated malware detection models: a survey](https://mlsanity.com/paper/2101.06232/) (2021)
- [Opening the random forest black box by the analysis of the mutual impact of features](https://mlsanity.com/paper/2304.02490/) (2023)
- [Effector: A Python package for regional explanations](https://mlsanity.com/paper/2404.02629/) (2024)
- [CHILLI: A data context-aware perturbation method for XAI](https://mlsanity.com/paper/2407.07521/) (2024)

## Abstract (arXiv)

The interpretation of feature importance in machine learning models is challenging when features are dependent. Permutation feature importance (PFI) ignores such dependencies, which can cause misleading interpretations due to extrapolation. A possible remedy is more advanced conditional PFI approaches that enable the assessment of feature importance conditional on all other features. Due to this shift in perspective and in order to enable correct interpretations, it is therefore important that the conditioning is transparent and humanly comprehensible. In this paper, we propose a new sampling mechanism for the conditional distribution based on permutations in conditional subgroups. As these subgroups are constructed using decision trees (transformation trees), the conditioning becomes inherently interpretable. This not only provides a simple and effective estimator of conditional PFI, but also local PFI estimates within the subgroups. In addition, we apply the conditional subgroups approach to partial dependence plots (PDP), a popular method for describing feature effects that can also suffer from extrapolation when features are dependent and interactions are present in the model. We show that PFI and PDP based on conditional subgroups often outperform methods such as conditional PFI based on knockoffs, or accumulated local effect plots. Furthermore, our approach allows for a more fine-grained interpretation of feature effects and importance within the conditional subgroups.