# Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach

arXiv: https://arxiv.org/abs/1606.09066 · HTML: https://mlsanity.com/paper/1606.09066/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jun 2016.

## Built on by (most cited first, 27 of 27)

- [CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms](https://mlsanity.com/paper/2108.00783/) (2021)
- [Fast Sparse Decision Tree Optimization via Reference Ensembles](https://mlsanity.com/paper/2112.00798/) (2021)
- [Assessing the communication gap between AI models and healthcare professionals: explainability, utility and trust in AI-driven clinical decision-making](https://mlsanity.com/paper/2204.05030/) (2022)
- [Learning Interpretable Rules for Scalable Data Representation and Classification](https://mlsanity.com/paper/2310.14336/) (2023)
- [Conclusive Local Interpretation Rules for Random Forests](https://mlsanity.com/paper/2104.06040/) (2021)
- [AcME -- Accelerated Model-agnostic Explanations: Fast Whitening of the Machine-Learning Black Box](https://mlsanity.com/paper/2112.12635/) (2021)
- [A Survey of Explainable Knowledge Tracing](https://mlsanity.com/paper/2403.07279/) (2024)
- [A Comprehensive Guide to Explainable AI: From Classical Models to LLMs](https://mlsanity.com/paper/2412.00800/) (2024)
- [Explainable AI applications in the Medical Domain: a systematic review](https://mlsanity.com/paper/2308.05411/) (2023)
- [Decision Predicate Graphs: Enhancing Interpretability in Tree Ensembles](https://mlsanity.com/paper/2404.02942/) (2024)
- [Generating Explainable Rule Sets from Tree-Ensemble Learning Methods by Answer Set Programming](https://mlsanity.com/paper/2109.08290/) (2021)
- [Improving the compromise between accuracy, interpretability and personalization of rule-based machine learning in medical problems](https://mlsanity.com/paper/2106.07827/) (2021)
- [Explainable Models via Compression of Tree Ensembles](https://mlsanity.com/paper/2206.07904/) (2022)
- [Local Multi-Label Explanations for Random Forest](https://mlsanity.com/paper/2207.01994/) (2022)
- [Forest-ORE: Mining Optimal Rule Ensemble to interpret Random Forest models](https://mlsanity.com/paper/2403.17588/) (2024)
- [A survey and taxonomy of methods interpreting random forest models](https://mlsanity.com/paper/2407.12759/) (2024)
- [BELLATREX: Building Explanations through a LocaLly AccuraTe Rule EXtractor](https://mlsanity.com/paper/2203.15511/) (2022)
- [Simplification of Forest Classifiers and Regressors](https://mlsanity.com/paper/2212.07103/) (2022)
- [RuleExplorer: A Scalable Matrix Visualization for Understanding Tree Ensemble Classifiers](https://mlsanity.com/paper/2409.03164/) (2024)
- [Generating Global and Local Explanations for Tree-Ensemble Learning Methods by Answer Set Programming](https://mlsanity.com/paper/2410.11000/) (2024)
- [HEX: Human-in-the-loop Explainability via Deep Reinforcement Learning](https://mlsanity.com/paper/2206.01343/) (2022)
- [TE2Rules: Explaining Tree Ensembles using Rules](https://mlsanity.com/paper/2206.14359/) (2022)
- [Explaining with Greater Support: Weighted Column Sampling Optimization for q-Consistent Summary-Explanations](https://mlsanity.com/paper/2302.04528/) (2023)
- [Combination of Weak Learners eXplanations to Improve Random Forest eXplicability Robustness](https://mlsanity.com/paper/2402.19025/) (2024)
- [Minimising changes to audit when updating decision trees](https://mlsanity.com/paper/2408.16321/) (2024)
- [Interpretable Responsibility Sharing as a Heuristic for Task and Motion Planning](https://mlsanity.com/paper/2409.05586/) (2024)
- [Learning High Coverage Discriminative Parsimonious Rulesets](https://mlsanity.com/paper/2606.14156/) (2026)

## Abstract (arXiv)

Tree ensembles, such as random forests and boosted trees, are renowned for their high prediction performance. However, their interpretability is critically limited due to the enormous complexity. In this study, we present a method to make a complex tree ensemble interpretable by simplifying the model. Specifically, we formalize the simplification of tree ensembles as a model selection problem. Given a complex tree ensemble, we aim at obtaining the simplest representation that is essentially equivalent to the original one. To this end, we derive a Bayesian model selection algorithm that optimizes the simplified model while maintaining the prediction performance. Our numerical experiments on several datasets showed that complicated tree ensembles were reasonably approximated as interpretable.