# High-Dimensional L2Boosting: Rate of Convergence

arXiv: https://arxiv.org/abs/1602.08927 · HTML: https://mlsanity.com/paper/1602.08927/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Feb 2016.

## Built on by (most cited first, 10 of 10)

- [Improving the Finite Sample Estimation of Average Treatment Effects using Double/Debiased Machine Learning with Propensity Score Calibration](https://mlsanity.com/paper/2409.04874/) (2024)
- [Lassoed Tree Boosting](https://mlsanity.com/paper/2205.10697/) (2022)
- [Double Machine Learning for Partially Linear Mixed-Effects Models with Repeated Measurements](https://mlsanity.com/paper/2108.13657/) (2021)
- [Calibrating doubly-robust estimators with unbalanced treatment assignment](https://mlsanity.com/paper/2403.01585/) (2024)
- [Generative Augmented Inference of LLM-generated Data for Market Research: Theory and Empirical Evidence](https://mlsanity.com/paper/2604.14575/) (2026)
- [Treatment Effect Estimation with Observational Network Data using Machine Learning](https://mlsanity.com/paper/2206.14591/) (2022)
- [ddml: Double/debiased machine learning in Stata](https://mlsanity.com/paper/2301.09397/) (2023)
- [Orthogonal Series Estimation for the Ratio of Conditional Expectation Functions](https://mlsanity.com/paper/2212.13145/) (2022)
- [Causal Machine Learning for Moderation Effects](https://mlsanity.com/paper/2401.08290/) (2024)
- [Targeted Learning for Variable Importance](https://mlsanity.com/paper/2411.02221/) (2024)

## Abstract (arXiv)

Boosting is one of the most significant developments in machine learning. This paper studies the rate of convergence of L2Boosting, which is tailored for regression, in a high-dimensional setting. Moreover, we introduce so-called textquotedblleft post-Boostingtextquotedblright. This is a post-selection estimator which applies ordinary least squares to the variables selected in the first stage by L2Boosting. Another variant is textquotedblleft Orthogonal Boostingtextquotedblright where after each step an orthogonal projection is conducted. We show that both post-L2Boosting and the orthogonal boosting achieve the same rate of convergence as LASSO in a sparse, high-dimensional setting. We show that the rate of convergence of the classical L2Boosting depends on the design matrix described by a sparse eigenvalue constant. To show the latter results, we derive new approximation results for the pure greedy algorithm, based on analyzing the revisiting behavior of L2Boosting. We also introduce feasible rules for early stopping, which can be easily implemented and used in applied work. Our results also allow a direct comparison between LASSO and boosting which has been missing from the literature. Finally, we present simulation studies and applications to illustrate the relevance of our theoretical results and to provide insights into the practical aspects of boosting. In these simulation studies, post-L2Boosting clearly outperforms LASSO.