mlsanityData to 7 Oct 2026

The Role of Depth, Width, and Tree Size in Expressiveness of Deep Forest

arXiv 2407.05108 ↗First posted 6 Jul 2024cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Random forests are classical ensemble algorithms that construct multiple randomized decision trees and aggregate their predictions using naive averaging. citetzhou2019deep further propose a deep forest algorithm with multi-layer forests, which outperforms random forests in various tasks. The performance of deep forests is related to three hyperparameters in practice: depth, width, and tree size, but little has been known about its theoretical explanation. This work provides the first upper and lower bounds on the approximation complexity of deep forests concerning the three hyperparameters. Our results confirm the distinctive role of depth, which can exponentially enhance the expressiveness of deep forests compared with width and tree size. Experiments confirm the theoretical findings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 49 references that are in the map. First posted on arXiv 6 Jul 2024.

Builds on

24 of 49 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  3. 2014On the Number of Linear Regions of Deep Neural Networkscited by 378
  4. 2019Interpretable machine learning: definitions, methods, and applicationscited by 303
  5. 2016Benefits of depth in neural networkscited by 248
  6. 2015The Power of Depth for Feedforward Neural Networkscited by 181
  7. 2014Consistency of random forestscited by 84
  8. 2017Depth Separation for Neural Networkscited by 46
  9. 2017Deep Forestcited by 46
  10. 2017Iterative Random Forests to detect predictive and stable high-order interactionscited by 42
  11. 2023Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuroncited by 25
  12. 2022On the Number of Regions of Piecewise Linear Neural Networkscited by 21
12 more
  1. 2013Narrowing the Gap: Random Forests In Theory and In Practicecited by 17
  2. 2024Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smootherscited by 13
  3. 2019Multi-Label Learning with Deep Forestcited by 12
  4. 2022Width is Less Important than Depth in ReLU Neural Networkscited by 10
  5. 2018Distributed Deep Forest and its Application to Automatic Detection of Cash-out Fraudcited by 7
  6. 2021Decision tree heuristics can fail, even in the smoothed settingcited by 7
  7. 2021Towards Understanding Theoretical Advantages of Complex-Reaction Networkscited by 6
  8. 2020Analyzing the tree-layer structure of Deep Forestscited by 5
  9. 2023On the Convergence of CART under Sufficient Impurity Decrease Conditioncited by 5
  10. 2023Stability of Random Forests and Coverage of Random-Forest Prediction Intervalscited by 4
  11. 2017Discriminative Metric Learning with Deep Forestcited by 3
  12. 2021Theoretical Exploration of Flexible Transmitter Modelcited by 3

Built on by

Not cited yet.