# Understanding Random Forests: From Theory to Practice

arXiv: https://arxiv.org/abs/1407.7502 · HTML: https://mlsanity.com/paper/1407.7502/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2014.

## Built on by (most cited first, 30 of 57)

- [Large Language Model Agent in Financial Trading: A Survey](https://mlsanity.com/paper/2408.06361/) (2024)
- [Large Scale Prediction with Decision Trees](https://mlsanity.com/paper/2104.13881/) (2021)
- [SHAFF: Fast and consistent SHApley eFfect estimates via random Forests](https://mlsanity.com/paper/2105.11724/) (2021)
- [Improving aircraft performance using machine learning: a review](https://mlsanity.com/paper/2210.11481/) (2022)
- [Quantitative Evaluation of Explainable Graph Neural Networks for Molecular Property Prediction](https://mlsanity.com/paper/2107.04119/) (2021)
- [QUANT: A Minimalist Interval Method for Time Series Classification](https://mlsanity.com/paper/2308.00928/) (2023)
- [Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictions](https://mlsanity.com/paper/2211.03524/) (2022)
- [MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA](https://mlsanity.com/paper/2102.13347/) (2021)
- [Comprehensive evaluation of Mal-API-2019 dataset by machine learning in malware detection](https://mlsanity.com/paper/2403.02232/) (2024)
- [A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic](https://mlsanity.com/paper/2204.06362/) (2022)
- [Machine learning-assisted surrogate construction for full-core fuel performance analysis](https://mlsanity.com/paper/2104.09499/) (2021)
- [Explainable Artificial Intelligence for Pharmacovigilance: What Features Are Important When Predicting Adverse Outcomes?](https://mlsanity.com/paper/2112.13210/) (2021)
- [Ground Truth Inference for Weakly Supervised Entity Matching](https://mlsanity.com/paper/2211.06975/) (2022)
- [Algorithm-Agnostic Explainability for Unsupervised Clustering](https://mlsanity.com/paper/2105.08053/) (2021)
- [From global to local MDI variable importances for random forests and when they are Shapley values](https://mlsanity.com/paper/2111.02218/) (2021)
- [On Uncertainty Estimation by Tree-based Surrogate Models in Sequential Model-based Optimization](https://mlsanity.com/paper/2202.10669/) (2022)
- [Multi-class Probabilistic Bounds for Self-learning](https://mlsanity.com/paper/2109.14422/) (2021)
- [Label Ranking through Nonparametric Regression](https://mlsanity.com/paper/2111.02749/) (2021)
- [Sky Imager-Based Forecast of Solar Irradiance Using Machine Learning](https://mlsanity.com/paper/2310.17356/) (2023)
- [ShapG: new feature importance method based on the Shapley value](https://mlsanity.com/paper/2407.00506/) (2024)
- [Classification algorithms applied to structure formation simulations](https://mlsanity.com/paper/2106.06587/) (2021)
- [Evaluation of physics constrained data-driven methods for turbulence model uncertainty quantification](https://mlsanity.com/paper/2210.00002/) (2022)
- [Explainable Artificial Intelligence for Improved Modeling of Processes](https://mlsanity.com/paper/2212.00695/) (2022)
- [Lightweight Online Learning for Sets of Related Problems in Automated Reasoning](https://mlsanity.com/paper/2305.11087/) (2023)
- [Prediction of Large Magnetic Moment Materials With Graph Neural Networks and Random Forests](https://mlsanity.com/paper/2111.14712/) (2021)
- [An Exploration of How Training Set Composition Bias in Machine Learning Affects Identifying Rare Objects](https://mlsanity.com/paper/2207.03207/) (2022)
- [Enhancing E-Commerce Recommendation using Pre-Trained Language Model and Fine-Tuning](https://mlsanity.com/paper/2302.04443/) (2023)
- [DynFrs: An Efficient Framework for Machine Unlearning in Random Forest](https://mlsanity.com/paper/2410.01588/) (2024)
- [Gungnir: Exploiting Stylistic Features in Images for Backdoor Attacks on Diffusion Models](https://mlsanity.com/paper/2502.20650/) (2025)
- [A Study on Inference Latency for Vision Transformers on Mobile Devices](https://mlsanity.com/paper/2510.25166/) (2025)

## Abstract (arXiv)

Data analysis and machine learning have become an integrative part of the modern scientific methodology, offering automated procedures for the prediction of a phenomenon based on past observations, unraveling underlying patterns in data and providing insights about the problem. Yet, caution should avoid using machine learning as a black-box tool, but rather consider it as a methodology, with a rational thought process that is entirely dependent on the problem under study. In particular, the use of algorithms should ideally require a reasonable understanding of their mechanisms, properties and limitations, in order to better apprehend and interpret their results. Accordingly, the goal of this thesis is to provide an in-depth analysis of random forests, consistently calling into question each and every part of the algorithm, in order to shed new light on its learning capabilities, inner workings and interpretability. The first part of this work studies the induction of decision trees and the construction of ensembles of randomized trees, motivating their design and purpose whenever possible. Our contributions follow with an original complexity analysis of random forests, showing their good computational performance and scalability, along with an in-depth discussion of their implementation details, as contributed within Scikit-Learn. In the second part of this work, we analyse and discuss the interpretability of random forests in the eyes of variable importance measures. The core of our contributions rests in the theoretical characterization of the Mean Decrease of Impurity variable importance measure, from which we prove and derive some of its properties in the case of multiway totally randomized trees and in asymptotic conditions. In consequence of this work, our analysis demonstrates that variable importances [...].