mlsanityData to 7 Oct 2026

On marginal feature attributions of tree-based models

arXiv 2302.08434 ↗First posted 16 Feb 2023cs.LG cs.GTcited by 3

Not on the map. Why?

Abstract (arXiv)

Due to their power and ease of use, tree-based machine learning models, such as random forests and gradient-boosted tree ensembles, have become very popular. To interpret them, local feature attributions based on marginal expectations, e.g. marginal (interventional) Shapley, Owen or Banzhaf values, may be employed. Such methods are true to the model and implementation invariant, i.e. dependent only on the input-output function of the model. We contrast this with the popular TreeSHAP algorithm by presenting two (statistically similar) decision trees that compute the exact same function for which the "path-dependent" TreeSHAP yields different rankings of features, whereas the marginal Shapley values coincide. Furthermore, we discuss how the internal structure of tree-based models may be leveraged to help with computing their marginal feature attributions according to a linear game value. One important observation is that these are simple (piecewise-constant) functions with respect to a certain grid partition of the input space determined by the trained model. Another crucial observation, showcased by experiments with XGBoost, LightGBM and CatBoost libraries, is that only a portion of all features appears in a tree from the ensemble. Thus, the complexity of computing marginal Shapley (or Owen or Banzhaf) feature attributions may be reduced. This remains valid for a broader class of game values which we shall axiomatically characterize. A prime example is the case of CatBoost models where the trees are oblivious (symmetric) and the number of features in each of them is no larger than the depth. We exploit the symmetry to derive an explicit formula, with improved complexity and only in terms of the internal model parameters, for marginal Shapley (and Banzhaf and Owen) values of CatBoost models. This results in a fast, accurate algorithm for estimating these feature attributions.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 83 references that are in the map. First posted on arXiv 16 Feb 2023.

Builds on

29 of 83 references.

References · 29

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  5. 2021Tabular Data: Deep Learning is Not All You Needcited by 530
  6. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  7. 2014On the Number of Linear Regions of Deep Neural Networkscited by 378
  8. 2019The many Shapley values for model explanationcited by 341
  9. 2016On the Expressive Power of Deep Neural Networkscited by 302
  10. 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 279
  11. 2019InterpretML: A Unified Framework for Machine Learning Interpretabilitycited by 243
  12. 2018CatBoost: gradient boosting with categorical features supportcited by 208
17 more
  1. 2019Explaining individual predictions when features are dependent: More accurate approximations to Shapley valuescited by 205
  2. 2019Feature relevance quantification in explainable AI: A causal problemcited by 164
  3. 2020True to the Model or True to the Data?cited by 123
  4. 2022Algorithms to estimate Shapley value feature attributionscited by 114
  5. 2020GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactionscited by 82
  6. 2017Interpretable & Explorable Approximations of Black Box Modelscited by 54
  7. 2018Explainable Neural Networks based on Additive Index Modelscited by 33
  8. 2020Unwrapping The Black Box of Deep ReLU Networks: Interpretability, Diagnostics, and Simplificationcited by 32
  9. 2019Enhancing Explainability of Neural Networks through Architecture Constraintscited by 29
  10. 2019Explaining Models by Propagating Shapley Values of Local Componentscited by 29
  11. 2021Fast TreeSHAP: Accelerating SHAP Value Computation for Treescited by 28
  12. 2021Designing Inherently Interpretable Machine Learning Modelscited by 23
  13. 2021groupShapley: Efficient prediction explanation with Shapley values for feature groupscited by 21
  14. 2021Accurate Shapley Values for explaining tree-based modelscited by 9
  15. 2020Supervised Machine Learning Techniques: An Overview with Applications to Bankingcited by 7
  16. 2021Self-interpretable Convolutional Neural Networks for Text Classificationcited by 5
  17. 2021Exact Shapley Values for Local and Model-True Explanations of Decision Tree Ensemblescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 1; 2026: 1.

3 papers.

  1. 2023Approximation of group explainers with coalition structure using Monte Carlo sampling on the product space of coalitions and featurescited by 2
  2. 2026Quadrature-TreeSHAP: Depth-Independent TreeSHAP and Shapley Interactionscited by 1
  3. 2025Explainable post-training bias mitigation with distribution-based fairness metricscited by 0