# AdaBoost and Forward Stagewise Regression are First-Order Convex Optimization Methods

arXiv: https://arxiv.org/abs/1307.1192 · HTML: https://mlsanity.com/paper/1307.1192/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jul 2013.

## Built on by (most cited first, 2 of 2)

- [AdaSelection: Accelerating Deep Learning Training through Data Subsampling](https://mlsanity.com/paper/2306.10728/) (2023)
- [Overview of AdaBoost : Reconciling its views to better understand its dynamics](https://mlsanity.com/paper/2310.18323/) (2023)

## Abstract (arXiv)

Boosting methods are highly popular and effective supervised learning methods which combine weak learners into a single accurate model with good statistical performance. In this paper, we analyze two well-known boosting methods, AdaBoost and Incremental Forward Stagewise Regression (FS_varepsilon), by establishing their precise connections to the Mirror Descent algorithm, which is a first-order method in convex optimization. As a consequence of these connections we obtain novel computational guarantees for these boosting methods. In particular, we characterize convergence bounds of AdaBoost, related to both the margin and log-exponential loss function, for any step-size sequence. Furthermore, this paper presents, for the first time, precise computational complexity results for FS_varepsilon.