# Online Gradient Boosting

arXiv: https://arxiv.org/abs/1506.04820 · HTML: https://mlsanity.com/paper/1506.04820/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2015.

## Built on by (most cited first, 15 of 15)

- [Ensemble deep learning: A review](https://mlsanity.com/paper/2104.02395/) (2021)
- [Adversarial Tracking Control via Strongly Adaptive Online Learning with Memory](https://mlsanity.com/paper/2102.01623/) (2021)
- [Boosting for Online Convex Optimization](https://mlsanity.com/paper/2102.09305/) (2021)
- [Dealing with Expert Bias in Collective Decision-Making](https://mlsanity.com/paper/2106.13539/) (2021)
- [Online Agnostic Multiclass Boosting](https://mlsanity.com/paper/2205.15113/) (2022)
- [Shrub Ensembles for Online Classification](https://mlsanity.com/paper/2112.03723/) (2021)
- [Online Residual Learning from Offline Experts for Pedestrian Tracking](https://mlsanity.com/paper/2409.04069/) (2024)
- [Minimax-optimal and Locally-adaptive Online Nonparametric Regression](https://mlsanity.com/paper/2410.03363/) (2024)
- [Online Gradient Boosting Decision Tree: In-Place Updates for Efficient Adding/Deleting Data](https://mlsanity.com/paper/2502.01634/) (2025)
- [Lift What You Can: Green Online Learning with Heterogeneous Ensembles](https://mlsanity.com/paper/2509.18962/) (2025)
- [URANUS: Radio Frequency Tracking, Classification and Identification of Unmanned Aircraft Vehicles](https://mlsanity.com/paper/2207.06025/) (2022)
- [The Nah Bandit: Modeling User Non-compliance in Recommendation Systems](https://mlsanity.com/paper/2408.07897/) (2024)
- [Fr'echet Geodesic Boosting](https://mlsanity.com/paper/2509.18013/) (2025)
- [CLeAN: Continual Learning Adaptive Normalization in Dynamic Environments](https://mlsanity.com/paper/2603.17548/) (2026)
- [Defensive Boosting for Online Probabilistic Forecasting](https://mlsanity.com/paper/2608.13554/) (2026)

## Abstract (arXiv)

We extend the theory of boosting for regression problems to the online learning setting. Generalizing from the batch setting for boosting, the notion of a weak learning algorithm is modeled as an online learning algorithm with linear loss functions that competes with a base class of regression functions, while a strong learning algorithm is an online learning algorithm with convex loss functions that competes with a larger class of regression functions. Our main result is an online gradient boosting algorithm which converts a weak online learning algorithm into a strong one where the larger class of functions is the linear span of the base class. We also give a simpler boosting algorithm that converts a weak online learning algorithm into a strong one where the larger class of functions is the convex hull of the base class, and prove its optimality.