# Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning

arXiv: https://arxiv.org/abs/2102.12328 · HTML: https://mlsanity.com/paper/2102.12328/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 69 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 25 of 25)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [A Random Forest Guided Tour](https://mlsanity.com/paper/1511.05741/) (2015)
- [Controlling the false discovery rate via knockoffs](https://mlsanity.com/paper/1404.5609/) (2014)
- [Consistency of random forests](https://mlsanity.com/paper/1405.2881/) (2014)
- [Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers](https://mlsanity.com/paper/1504.07676/) (2015)
- [Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead](https://mlsanity.com/paper/1811.10154/) (2018)
- [Efficient Variational Inference for Sparse Deep Learning with Theoretical Guarantee](https://mlsanity.com/paper/2011.07439/) (2020)
- [Unbiased Measurement of Feature Importance in Tree-Based Methods](https://mlsanity.com/paper/1903.05179/) (2019)
- [Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success](https://mlsanity.com/paper/1911.00190/) (2019)
- [A Debiased MDI Feature Importance Measure for Random Forests](https://mlsanity.com/paper/1906.10845/) (2019)
- [Approximation and Estimation for High-Dimensional Deep Learning Networks](https://mlsanity.com/paper/1809.03090/) (2018)
- [Fast and Powerful Conditional Randomization Testing via Distillation](https://mlsanity.com/paper/2006.03980/) (2020)
- [Asymptotic Distributions and Rates of Convergence for Random Forests via Generalized U-statistics](https://mlsanity.com/paper/1905.10651/) (2019)
- [On Theory for BART](https://mlsanity.com/paper/1810.00787/) (2018)
- [Trees, forests, and impurity-based variable importance](https://mlsanity.com/paper/2001.04295/) (2020)
- [Unbiased variable importance for random forests](https://mlsanity.com/paper/2003.02106/) (2020)
- [Getting Better from Worse: Augmented Bagging and a Cautionary Tale of Variable Importance](https://mlsanity.com/paper/2003.03629/) (2020)
- [Scalable and Efficient Hypothesis Testing with Random Forests](https://mlsanity.com/paper/1904.07830/) (2019)
- [A Unified Framework for Random Forest Prediction Error Estimation](https://mlsanity.com/paper/1912.07435/) (2019)
- [Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution](https://mlsanity.com/paper/1806.09762/) (2018)
- [RAPTT: An Exact Two-Sample Test in High Dimensions Using Random Projections](https://mlsanity.com/paper/1405.1792/) (2014)
- [The Holdout Randomization Test for Feature Selection in Black Box Models](https://mlsanity.com/paper/1811.00645/) (2018)

## Built on by (most cited first, 7 of 7)

- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [Hierarchical Shrinkage: improving the accuracy and interpretability of tree-based methods](https://mlsanity.com/paper/2202.00858/) (2022)
- [A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds](https://mlsanity.com/paper/2110.09626/) (2021)
- [A Statistical-Modelling Approach to Feedforward Neural Network Model Selection](https://mlsanity.com/paper/2207.04248/) (2022)
- [Comparison of generalised additive models and neural networks in applications: A systematic review](https://mlsanity.com/paper/2510.24601/) (2025)
- [Do We Really Even Need Data? A Modern Look at Drawing Inference with Predicted Data](https://mlsanity.com/paper/2512.05456/) (2025)
- [Investigating Statistical Inference and Covariate Effects in Shallow Neural Networks](https://mlsanity.com/paper/2311.08139/) (2023)

## Abstract (arXiv)

In 2001, Leo Breiman wrote of a divide between "data modeling" and "algorithmic modeling" cultures. Twenty years later this division feels far more ephemeral, both in terms of assigning individuals to camps, and in terms of intellectual boundaries. We argue that this is largely due to the "data modelers" incorporating algorithmic methods into their toolbox, particularly driven by recent developments in the statistical understanding of Breiman's own Random Forest methods. While this can be simplistically described as "Breiman won", these same developments also expose the limitations of the prediction-first philosophy that he espoused, making careful statistical analysis all the more important. This paper outlines these exciting recent developments in the random forest literature which, in our view, occurred as a result of a necessary blending of the two ways of thinking Breiman originally described. We also ask what areas statistics and statisticians might currently overlook.