# An exact counterfactual-example-based approach to tree-ensemble models interpretability

arXiv: https://arxiv.org/abs/2105.14820 · HTML: https://mlsanity.com/paper/2105.14820/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 23 of 23)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [Counterfactual Visual Explanations](https://mlsanity.com/paper/1904.07451/) (2019)
- [Model-Agnostic Counterfactual Explanations for Consequential Decisions](https://mlsanity.com/paper/1905.11190/) (2019)
- [Local Rule-Based Explanations of Black Box Decision Systems](https://mlsanity.com/paper/1805.10820/) (2018)
- [Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model](https://mlsanity.com/paper/1511.01644/) (2015)
- [Explaining Explanations in AI](https://mlsanity.com/paper/1811.01439/) (2018)
- [If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques](https://mlsanity.com/paper/2103.01035/) (2021)
- [Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking](https://mlsanity.com/paper/1706.06691/) (2017)
- [Measurable Counterfactual Local Explanations for Any Classifier](https://mlsanity.com/paper/1908.03020/) (2019)
- [Interpretable Credit Application Predictions With Counterfactual Explanations](https://mlsanity.com/paper/1811.05245/) (2018)
- [FOCUS: Flexible Optimizable Counterfactual Explanations for Tree Ensembles](https://mlsanity.com/paper/1911.12199/) (2019)
- [Explaining Deep Learning Models with Constrained Adversarial Examples](https://mlsanity.com/paper/1906.10671/) (2019)
- [Counterfactual Explanations for Oblique Decision Trees: Exact, Efficient Algorithms](https://mlsanity.com/paper/2103.01096/) (2021)
- [Plausible Counterfactuals: Auditing Deep Learning Classifiers with Realistic Adversarial Examples](https://mlsanity.com/paper/2003.11323/) (2020)
- [Making Tree Ensembles Interpretable](https://mlsanity.com/paper/1606.05390/) (2016)
- [Interpreting extracted rules from ensemble of trees: Application to computer-aided diagnosis of breast MRI](https://mlsanity.com/paper/1606.08288/) (2016)
- [LEAFAGE: Example-based and Feature importance-based Explanationsfor Black-box ML models](https://mlsanity.com/paper/1812.09044/) (2018)

## Built on by (most cited first, 3 of 3)

- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [A survey and taxonomy of methods interpreting random forest models](https://mlsanity.com/paper/2407.12759/) (2024)
- [Counterfactual Maps: What They Are and How to Find Them](https://mlsanity.com/paper/2602.09128/) (2026)

## Abstract (arXiv)

Explaining the decisions of machine learning models is becoming a necessity in many areas where trust in ML models decision is key to their accreditation/adoption. The ability to explain models decisions also allows to provide diagnosis in addition to the model decision, which is highly valuable in scenarios such as fault detection. Unfortunately, high-performance models do not exhibit the necessary transparency to make their decisions fully understandable. And the black-boxes approaches, which are used to explain such model decisions, suffer from a lack of accuracy in tracing back the exact cause of a model decision regarding a given input. Indeed, they do not have the ability to explicitly describe the decision regions of the model around that input, which is necessary to determine what influences the model towards one decision or the other. We thus asked ourselves the question: is there a category of high-performance models among the ones currently used for which we could explicitly and exactly characterise the decision regions in the input feature space using a geometrical characterisation? Surprisingly we came out with a positive answer for any model that enters the category of tree ensemble models, which encompasses a wide range of high-performance models such as XGBoost, LightGBM, random forests ... We could derive an exact geometrical characterisation of their decision regions under the form of a collection of multidimensional intervals. This characterisation makes it straightforward to compute the optimal counterfactual (CF) example associated with a query point. We demonstrate several possibilities of the approach, such as computing the CF example based only on a subset of features. This allows to obtain more plausible explanations by adding prior knowledge about which variables the user can control. An adaptation to CF reasoning on regression problems is also envisaged.