# Explainable AI for Trees: From Local Explanations to Global Understanding

arXiv: https://arxiv.org/abs/1905.04610 · HTML: https://mlsanity.com/paper/1905.04610/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2019.

## Built on by (most cited first, 30 of 65)

- [Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2110.14049/) (2021)
- [Explaining black box text modules in natural language with language models](https://mlsanity.com/paper/2305.09863/) (2023)
- [Shapley values for feature selection: The good, the bad, and the axioms](https://mlsanity.com/paper/2102.10936/) (2021)
- [Augmenting Interpretable Models with LLMs during Training](https://mlsanity.com/paper/2209.11799/) (2022)
- [Explaining a Series of Models by Propagating Shapley Values](https://mlsanity.com/paper/2105.00108/) (2021)
- [Fast TreeSHAP: Accelerating SHAP Value Computation for Trees](https://mlsanity.com/paper/2109.09847/) (2021)
- [GLOBE-CE: A Translation-Based Approach for Global Counterfactual Explanations](https://mlsanity.com/paper/2305.17021/) (2023)
- [Best of both worlds: local and global explanations with human-understandable concepts](https://mlsanity.com/paper/2106.08641/) (2021)
- [Fast Interpretable Greedy-Tree Sums](https://mlsanity.com/paper/2201.11931/) (2022)
- [Feature Synergy, Redundancy, and Independence in Global Model Explanations using SHAP Vector Decomposition](https://mlsanity.com/paper/2107.12436/) (2021)
- [Monitoring Model Deterioration with Explainable Uncertainty Estimation via Non-parametric Bootstrap](https://mlsanity.com/paper/2201.11676/) (2022)
- [Evaluating the Correctness of Explainable AI Algorithms for Classification](https://mlsanity.com/paper/2105.09740/) (2021)
- [A k-additive Choquet integral-based approach to approximate the SHAP values for local interpretability in machine learning](https://mlsanity.com/paper/2211.02166/) (2022)
- [Fairness Implications of Encoding Protected Categorical Attributes](https://mlsanity.com/paper/2201.11358/) (2022)
- [Explanation Shift: Detecting distribution shifts on tabular data via the explanation space](https://mlsanity.com/paper/2210.12369/) (2022)
- [Efficient learning of large sets of locally optimal classification rules](https://mlsanity.com/paper/2301.09936/) (2023)
- [Confident Feature Ranking](https://mlsanity.com/paper/2307.15361/) (2023)
- [Desiderata for Explainable AI in statistical production systems of the European Central Bank](https://mlsanity.com/paper/2107.08045/) (2021)
- [Fighting Money Laundering with Statistics and Machine Learning](https://mlsanity.com/paper/2201.04207/) (2022)
- [XAudit : A Theoretical Look at Auditing with Explanations](https://mlsanity.com/paper/2206.04740/) (2022)
- [Explainable AI applications in the Medical Domain: a systematic review](https://mlsanity.com/paper/2308.05411/) (2023)
- [Explainable Artificial Intelligence (XAI) for Malware Analysis: A Survey of Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2409.13723/) (2024)
- [What Makes a Scientific Paper be Accepted for Publication?](https://mlsanity.com/paper/2104.07112/) (2021)
- [Pitfalls of Explainable ML: An Industry Perspective](https://mlsanity.com/paper/2106.07758/) (2021)
- [Interpretable Factor Decomposition for Decision Intelligence in Large-Scale Financial Markets: Evidence from China's A-Share Market](https://mlsanity.com/paper/2606.12843/) (2026)
- [Comparing Baseline Shapley and Integrated Gradients for Local Explanation: Some Additional Insights](https://mlsanity.com/paper/2208.06096/) (2022)
- [Feature Selection with Distance Correlation](https://mlsanity.com/paper/2212.00046/) (2022)
- [KnAC: an approach for enhancing cluster analysis with background knowledge and explanations](https://mlsanity.com/paper/2112.08759/) (2021)
- [MLHOps: Machine Learning for Healthcare Operations](https://mlsanity.com/paper/2305.02474/) (2023)
- [Detecting new obfuscated malware variants: A lightweight and interpretable machine learning approach](https://mlsanity.com/paper/2407.07918/) (2024)

## Abstract (arXiv)

Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are the most popular non-linear predictive models used in practice today, yet comparatively little attention has been paid to explaining their predictions. Here we significantly improve the interpretability of tree-based models through three main contributions: 1) The first polynomial time algorithm to compute optimal explanations based on game theory. 2) A new type of explanation that directly measures local feature interaction effects. 3) A new set of tools for understanding global model structure based on combining many local explanations of each prediction. We apply these tools to three medical machine learning problems and show how combining many high-quality local explanations allows us to represent global structure while retaining local faithfulness to the original model. These tools enable us to i) identify high magnitude but low frequency non-linear mortality risk factors in the general US population, ii) highlight distinct population sub-groups with shared risk characteristics, iii) identify non-linear interaction effects among risk factors for chronic kidney disease, and iv) monitor a machine learning model deployed in a hospital by identifying which features are degrading the model's performance over time. Given the popularity of tree-based machine learning models, these improvements to their interpretability have implications across a broad set of domains.