# A survey and taxonomy of methods interpreting random forest models

arXiv: https://arxiv.org/abs/2407.12759 · HTML: https://mlsanity.com/paper/2407.12759/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 92 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 30 of 34)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [A Random Forest Guided Tour](https://mlsanity.com/paper/1511.05741/) (2015)
- [Local Rule-Based Explanations of Black Box Decision Systems](https://mlsanity.com/paper/1805.10820/) (2018)
- [Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities](https://mlsanity.com/paper/2111.06420/) (2021)
- [Explainable AI for Trees: From Local Explanations to Global Understanding](https://mlsanity.com/paper/1905.04610/) (2019)
- [Interpreting Blackbox Models via Model Extraction](https://mlsanity.com/paper/1705.08504/) (2017)
- [Interpreting Tree Ensembles with inTrees](https://mlsanity.com/paper/1408.5456/) (2014)
- [Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach](https://mlsanity.com/paper/1606.09066/) (2016)
- [Explainable Matrix -- Visualization for Global and Local Interpretability of Random Forest Classification Ensembles](https://mlsanity.com/paper/2005.04289/) (2020)
- [Why model why? Assessing the strengths and limitations of LIME](https://mlsanity.com/paper/2012.00093/) (2020)
- [Survey of explainable machine learning with visual and granular methods beyond quasi-explanations](https://mlsanity.com/paper/2009.10221/) (2020)
- [Explainable Artificial Intelligence and Machine Learning: A reality rooted perspective](https://mlsanity.com/paper/2001.09464/) (2020)
- [What is Interpretable? Using Machine Learning to Design Interpretable Decision-Support Systems](https://mlsanity.com/paper/1811.10799/) (2018)
- [EUCA: the End-User-Centered Explainable AI Framework](https://mlsanity.com/paper/2102.02437/) (2021)
- [Conclusive Local Interpretation Rules for Random Forests](https://mlsanity.com/paper/2104.06040/) (2021)
- [Ensembles of Random SHAPs](https://mlsanity.com/paper/2103.03302/) (2021)
- [Identifying Explanation Needs of End-users: Applying and Extending the XAI Question Bank](https://mlsanity.com/paper/2307.09369/) (2023)
- [Interpreting random forest classification models using a feature contribution method](https://mlsanity.com/paper/1312.1121/) (2013)
- [A Parallel Random Forest Algorithm for Big Data in a Spark Cloud Computing Environment](https://mlsanity.com/paper/1810.07748/) (2018)
- [Rule Covering for Interpretation and Boosting](https://mlsanity.com/paper/2007.06379/) (2020)
- [TREX: Tree-Ensemble Representer-Point Explanations](https://mlsanity.com/paper/2009.05530/) (2020)
- [Extracting Optimal Explanations for Ensemble Trees via Logical Reasoning](https://mlsanity.com/paper/2103.02191/) (2021)
- [From unbiased MDI Feature Importance to Explainable AI for Trees](https://mlsanity.com/paper/2003.12043/) (2020)
- [Forest Guided Smoothing](https://mlsanity.com/paper/2103.05092/) (2021)
- [An exact counterfactual-example-based approach to tree-ensemble models interpretability](https://mlsanity.com/paper/2105.14820/) (2021)
- [Visualizing Random Forest with Self-Organising Map](https://mlsanity.com/paper/1405.6684/) (2014)

## Built on by (most cited first, 2 of 2)

- [TreeNet: Layered Decision Ensembles](https://mlsanity.com/paper/2510.09654/) (2025)
- [Kernel of Partition Paths: A Unified Representation for Tree Ensembles](https://mlsanity.com/paper/2606.18853/) (2026)

## Abstract (arXiv)

The interpretability of random forest (RF) models is a research topic of growing interest in the machine learning (ML) community. In the state of the art, RF is considered a powerful learning ensemble given its predictive performance, flexibility, and ease of use. Furthermore, the inner process of the RF model is understandable because it uses an intuitive and intelligible approach for building the RF decision tree ensemble. However, the RF resulting model is regarded as a "black box" because of its numerous deep decision trees. Gaining visibility over the entire process that induces the final decisions by exploring each decision tree is complicated, if not impossible. This complexity limits the acceptance and implementation of RF models in several fields of application. Several papers have tackled the interpretation of RF models. This paper aims to provide an extensive review of methods used in the literature to interpret RF resulting models. We have analyzed these methods and classified them based on different axes. Although this review is not exhaustive, it provides a taxonomy of various techniques that should guide users in choosing the most appropriate tools for interpreting RF models, depending on the interpretability aspects sought. It should also be valuable for researchers who aim to focus their work on the interpretability of RF or ML black boxes in general.