mlsanityData to 7 Oct 2026

Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning

arXiv 2102.12328 ↗First posted 23 Feb 2021stat.OT cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

In 2001, Leo Breiman wrote of a divide between "data modeling" and "algorithmic modeling" cultures. Twenty years later this division feels far more ephemeral, both in terms of assigning individuals to camps, and in terms of intellectual boundaries. We argue that this is largely due to the "data modelers" incorporating algorithmic methods into their toolbox, particularly driven by recent developments in the statistical understanding of Breiman's own Random Forest methods. While this can be simplistically described as "Breiman won", these same developments also expose the limitations of the prediction-first philosophy that he espoused, making careful statistical analysis all the more important. This paper outlines these exciting recent developments in the random forest literature which, in our view, occurred as a result of a necessary blending of the two ways of thinking Breiman originally described. We also ask what areas statistics and statisticians might currently overlook.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 25 of its 69 references that are in the map. First posted on arXiv 23 Feb 2021.

Builds on

25 of 69 references.

References · 25

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  3. 2015Estimation and Inference of Heterogeneous Treatment Effects using Random Forestscited by 597
  4. 2016Generalized Random Forestscited by 309
  5. 2015A Random Forest Guided Tourcited by 168
  6. 2014Controlling the false discovery rate via knockoffscited by 97
  7. 2014Consistency of random forestscited by 84
  8. 2015Explaining the Success of AdaBoost and Random Forests as Interpolating Classifierscited by 45
  9. 2018Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Insteadcited by 40
  10. 2020Efficient Variational Inference for Sparse Deep Learning with Theoretical Guaranteecited by 37
  11. 2019Unbiased Measurement of Feature Importance in Tree-Based Methodscited by 30
  12. 2019Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Successcited by 30
13 more
  1. 2019A Debiased MDI Feature Importance Measure for Random Forestscited by 24
  2. 2018Approximation and Estimation for High-Dimensional Deep Learning Networkscited by 22
  3. 2020Fast and Powerful Conditional Randomization Testing via Distillationcited by 19
  4. 2019Asymptotic Distributions and Rates of Convergence for Random Forests via Generalized U-statisticscited by 18
  5. 2018On Theory for BARTcited by 17
  6. 2020Trees, forests, and impurity-based variable importancecited by 16
  7. 2020Unbiased variable importance for random forestscited by 11
  8. 2020Getting Better from Worse: Augmented Bagging and a Cautionary Tale of Variable Importancecited by 10
  9. 2019Scalable and Efficient Hypothesis Testing with Random Forestscited by 8
  10. 2019A Unified Framework for Random Forest Prediction Error Estimationcited by 8
  11. 2018Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distributioncited by 7
  12. 2014RAPTT: An Exact Two-Sample Test in High Dimensions Using Random Projectionscited by 4
  13. 2018The Holdout Randomization Test for Feature Selection in Black Box Modelscited by 3

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 1; 2024: 0; 2025: 2.

7 papers.

  1. 2022Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomenacited by 21
  2. 2022Hierarchical Shrinkage: improving the accuracy and interpretability of tree-based methodscited by 18
  3. 2021A cautionary tale on fitting decision trees to data from additive models: generalization lower boundscited by 9
  4. 2022A Statistical-Modelling Approach to Feedforward Neural Network Model Selectioncited by 5
  5. 2025Comparison of generalised additive models and neural networks in applications: A systematic reviewcited by 3
  6. 2025Do We Really Even Need Data? A Modern Look at Drawing Inference with Predicted Datacited by 2
  7. 2023Investigating Statistical Inference and Covariate Effects in Shallow Neural Networkscited by 1