# Slow-Growing Trees

arXiv: https://arxiv.org/abs/2103.01926 · HTML: https://mlsanity.com/paper/2103.01926/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 49 references that are in the map.
- First posted on arXiv 2 Mar 2021.

## Builds on (most cited first, 11 of 11)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Optimal Sparse Decision Trees](https://mlsanity.com/paper/1904.12847/) (2019)
- [Optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularization](https://mlsanity.com/paper/1805.10939/) (2018)
- [Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers](https://mlsanity.com/paper/1504.07676/) (2015)
- [How is Machine Learning Useful for Macroeconomic Forecasting?](https://mlsanity.com/paper/2008.12477/) (2020)
- [Efficient non-greedy optimization of decision trees](https://mlsanity.com/paper/1511.04056/) (2015)
- [The Macroeconomy as a Random Forest](https://mlsanity.com/paper/2006.12724/) (2020)
- [Macroeconomic Data Transformations Matter](https://mlsanity.com/paper/2008.01714/) (2020)
- [Can Machine Learning Catch the COVID-19 Recession?](https://mlsanity.com/paper/2103.01201/) (2021)
- [To Bag is to Prune](https://mlsanity.com/paper/2008.07063/) (2020)

## Abstract (arXiv)

Random Forest's performance can be matched by a single slow-growing tree (SGT), which uses a learning rate to tame CART's greedy algorithm. SGT exploits the view that CART is an extreme case of an iterative weighted least square procedure. Moreover, a unifying view of Boosted Trees (BT) and Random Forests (RF) is presented. Greedy ML algorithms' outcomes can be improved using either "slow learning" or diversification. SGT applies the former to estimate a single deep tree, and Booging (bagging stochastic BT with a high learning rate) uses the latter with additive shallow trees. The performance of this tree ensemble quaternity (Booging, BT, SGT, RF) is assessed on simulated and real regression tasks.