# Lassoed Tree Boosting

arXiv: https://arxiv.org/abs/2205.10697 · HTML: https://mlsanity.com/paper/2205.10697/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 33 references that are in the map.
- First posted on arXiv 22 May 2022.

## Builds on (most cited first, 6 of 6)

- [Implicit Gradient Regularization](https://mlsanity.com/paper/2009.11162/) (2020)
- [Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networks](https://mlsanity.com/paper/1904.13262/) (2019)
- [Kaggle forecasting competitions: An overlooked learning opportunity](https://mlsanity.com/paper/2009.07701/) (2020)
- [Efficient Estimation of Pathwise Differentiable Target Parameters with the Undersmoothed Highly Adaptive Lasso](https://mlsanity.com/paper/1908.05607/) (2019)
- [Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation](https://mlsanity.com/paper/1903.01395/) (2019)
- [High-Dimensional L2Boosting: Rate of Convergence](https://mlsanity.com/paper/1602.08927/) (2016)

## Built on by (most cited first, 4 of 4)

- [Highly Adaptive Ridge](https://mlsanity.com/paper/2410.02680/) (2024)
- [Multi-task Highly Adaptive Lasso](https://mlsanity.com/paper/2301.12029/) (2023)
- [Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE](https://mlsanity.com/paper/2605.15483/) (2026)
- [Seeing the Forest for the Trees: The Gaussian Process Limit of BART](https://mlsanity.com/paper/2607.28844/) (2026)

## Abstract (arXiv)

Gradient boosting performs exceptionally in most prediction problems and scales well to large datasets. In this paper we prove that a ``lassoed'' gradient boosted tree algorithm with early stopping achieves faster than n-1/4 L2 convergence in the large nonparametric space of cadlag functions of bounded sectional variation. This rate is remarkable because it does not depend on the dimension, sparsity, or smoothness. We use simulation and real data to confirm our theory and demonstrate empirical performance and scalability on par with standard boosting. Our convergence proofs are based on a novel, general theorem on early stopping with empirical loss minimizers of nested Donsker classes.