mlsanityData to 14 Oct 2026

Large Scale Prediction with Decision Trees

arXiv 2104.13881 ↗First posted 28 Apr 2021stat.ML cs.LG math.STcited by 28

Not on the map. Why?

Abstract (arXiv)

This paper shows that decision trees constructed with Classification and Regression Trees (CART) and C4.5 methodology are consistent for regression and classification tasks, even when the number of predictor variables grows sub-exponentially with the sample size, under natural 0-norm and 1-norm sparsity constraints. The theory applies to a wide range of models, including (ordinary or logistic) additive regression models with component functions that are continuous, of bounded variation, or, more generally, Borel measurable. Consistency holds for arbitrary joint distributions of the predictor variables, thereby accommodating continuous, discrete, and/or dependent data. Finally, we show that these qualitative properties of individual trees are inherited by Breiman's random forests. A key step in the analysis is the establishment of an oracle inequality, which allows for a precise characterization of the goodness-of-fit and complexity tradeoff for a mis-specified model.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 13 of its 48 references that are in the map. First posted on arXiv 28 Apr 2021.

Builds on

13 of 48 references.

References · 13

  1. 2015Estimation and Inference of Heterogeneous Treatment Effects using Random Forestscited by 600
  2. 2014Consistency of random forestscited by 85
  3. 2018Hyperparameters and Tuning Strategies for Random Forestcited by 84
  4. 2014Understanding Random Forests: From Theory to Practicecited by 57
  5. 2019Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Successcited by 30
  6. 2017High-dimensional classification by sparse logistic regressioncited by 19
  7. 2018Minimax optimal rates for Mondrian trees and forestscited by 18
  8. 2013Narrowing the Gap: Random Forests In Theory and In Practicecited by 17
  9. 2019AMF: Aggregated Mondrian Forests for Online Learningcited by 17
  10. 2020Estimation and Inference with Trees and Forests in High Dimensionscited by 14
  11. 2021Trees, Forests, Chickens, and Eggs: When and Why to Prune Trees in a Random Forestcited by 13
  12. 2020Sparse learning with CARTcited by 12
1 more
  1. 2019Adaptive Estimation of Multivariate Piecewise Polynomials and Bounded Variation Functions by Optimal Decision Treescited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 4; 2024: 7; 2025: 11; 2026: 4.

28 papers.

Sort:

  1. 2023Adaptive Conformal Prediction by Reweighting Nonconformity Scorecited by 16
  2. 2023Synthetic Combinations: A Causal Inference Framework for Combinatorial Interventionscited by 12
  3. 2023Inference with Mondrian Random Forestscited by 6
  4. 2023On the Convergence of CART under Sufficient Impurity Decrease Conditioncited by 5
  5. 2021Minimax Rates for High-Dimensional Random Tessellation Forestscited by 4
  6. 2025Local MDI+: Local Feature Importances for Tree-Based Modelscited by 3
  7. 2022FACT: High-Dimensional Random Forests Inferencecited by 1
  8. 2024When does Subagging Work?cited by 1
  9. 2024Statistical Advantages of Oblique Randomized Decision Trees and Forestscited by 1
  10. 2024Efficient Decision Trees for Tensor Regressionscited by 1
  11. 2024Ranking Perspective for Tree-based Methods with Applications to Symbolic Feature Selectioncited by 1
  12. 2025EarlyStopping: Implicit Regularization for Iterative Learning Procedures in Pythoncited by 1
16 more
  1. 2025Revisiting Randomization in Greedy Model Searchcited by 1
  2. 2025Prospective Learning in Retrospectcited by 1
  3. 2024Analyze Additive and Interaction Effects via Collaborative Treescited by 0
  4. 2024Pure interaction effects unseen by Random Forestscited by 0
  5. 2024Consistency for Large Neural Networks: Regression and Classificationcited by 0
  6. 2025Analyzing decision tree bias towards the minority classcited by 0
  7. 2025FairUDT: Fairness-aware Uplift Decision Treescited by 0
  8. 2025Uniform convergence of the smooth calibration error and its relationship with functional gradientcited by 0
  9. 2025Earthquake Damage Grades Prediction using An Ensemble Approach Integrating Advanced Machine and Deep Learning Modelscited by 0
  10. 2025Transfer Learning with Distance Covariance for Random Forest: Error Bounds and an EHR Applicationcited by 0
  11. 2025Empowering Decision Trees via Shape Function Branchingcited by 0
  12. 2025Provable Recovery of Locally Important Signed Features and Interactions from Random Forestcited by 0
  13. 2026Covariance-Driven Regression Trees: Reducing Overfitting in CARTcited by 0
  14. 2026Minimax Rates and Spectral Distillation for Tree Ensemblescited by 0
  15. 2026Decision Tree Learning on Product Spacescited by 0
  16. 2026Kernel of Partition Paths: A Unified Representation for Tree Ensemblescited by 0