# Grouped Feature Importance and Combined Features Effect Plot

arXiv: https://arxiv.org/abs/2104.11688 · HTML: https://mlsanity.com/paper/2104.11688/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 108 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models](https://mlsanity.com/paper/2007.04131/) (2020)
- [Efficient nonparametric statistical inference on population feature importance using Shapley values](https://mlsanity.com/paper/2006.09481/) (2020)
- [Visualizing the Feature Importance for Black Box Models](https://mlsanity.com/paper/1804.06620/) (2018)
- [Kernel-based Tests for Joint Independence](https://mlsanity.com/paper/1603.00285/) (2016)
- [Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approach](https://mlsanity.com/paper/2006.04628/) (2020)
- [Testing Conditional Independence in Supervised Learning Algorithms](https://mlsanity.com/paper/1901.09917/) (2019)
- [Grouped variable importance with random forests and application to multiple functional data analysis](https://mlsanity.com/paper/1411.4170/) (2014)
- [Sampling, Intervention, Prediction, Aggregation: A Generalized Framework for Model-Agnostic Interpretations](https://mlsanity.com/paper/1904.03959/) (2019)
- [Human-interpretable model explainability on high-dimensional data](https://mlsanity.com/paper/2010.07384/) (2020)
- [The Shapley Value of coalition of variables provides better explanations](https://mlsanity.com/paper/2103.13342/) (2021)
- [Structured nonlinear variable selection](https://mlsanity.com/paper/1805.06258/) (2018)

## Built on by (most cited first, 14 of 14)

- [Comparing Explanation Methods for Traditional Machine Learning Models Part 1: An Overview of Current Methods and Quantifying Their Disagreement](https://mlsanity.com/paper/2211.08943/) (2022)
- [A Guide to Feature Importance Methods for Scientific Inference](https://mlsanity.com/paper/2404.12862/) (2024)
- [Unifying Feature-Based Explanations with Functional ANOVA and Cooperative Game Theory](https://mlsanity.com/paper/2412.17152/) (2024)
- [Conditional Feature Importance for Mixed Data](https://mlsanity.com/paper/2210.03047/) (2022)
- [Interpretable machine learning for time-to-event prediction in medicine and healthcare](https://mlsanity.com/paper/2303.09817/) (2023)
- [shapr: Explaining Machine Learning Models with Conditional Shapley Values in R and Python](https://mlsanity.com/paper/2504.01842/) (2025)
- [Variable Importance in High-Dimensional Settings Requires Grouping](https://mlsanity.com/paper/2312.10858/) (2023)
- [Explaining and visualizing black-box models through counterfactual paths](https://mlsanity.com/paper/2307.07764/) (2023)
- [The Most Important Features in Generalized Additive Models Might Be Groups of Features](https://mlsanity.com/paper/2506.19937/) (2025)
- [GRANITE: A Generalized Regional Framework for Identifying Agreement in Feature-Based Explanations](https://mlsanity.com/paper/2601.22771/) (2026)
- [xplainfi: Feature Importance and Statistical Inference for Machine Learning in R](https://mlsanity.com/paper/2603.15306/) (2026)
- [Explainable Modeling for Wind Power Forecasting: A Glass-Box Approach with High Accuracy](https://mlsanity.com/paper/2310.18629/) (2023)
- [Improving the Accuracy and Interpretability of Neural Networks for Wind Power Forecasting](https://mlsanity.com/paper/2312.15741/) (2023)
- [Beyond Additivity: Sparse Isotonic Shapley Regression toward Nonlinear Explainability](https://mlsanity.com/paper/2512.03112/) (2025)

## Abstract (arXiv)

Interpretable machine learning has become a very active area of research due to the rising popularity of machine learning algorithms and their inherently challenging interpretability. Most work in this area has been focused on the interpretation of single features in a model. However, for researchers and practitioners, it is often equally important to quantify the importance or visualize the effect of feature groups. To address this research gap, we provide a comprehensive overview of how existing model-agnostic techniques can be defined for feature groups to assess the grouped feature importance, focusing on permutation-based, refitting, and Shapley-based methods. We also introduce an importance-based sequential procedure that identifies a stable and well-performing combination of features in the grouped feature space. Furthermore, we introduce the combined features effect plot, which is a technique to visualize the effect of a group of features based on a sparse, interpretable linear combination of features. We used simulation studies and a real data example from computational psychology to analyze, compare, and discuss these methods.