# Infinitesimal gradient boosting

arXiv: https://arxiv.org/abs/2104.13208 · HTML: https://mlsanity.com/paper/2104.13208/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 25 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 3 of 3)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Optimization by gradient boosting](https://mlsanity.com/paper/1707.05023/) (2017)
- [Behavior of linear L2-boosting algorithms in the vanishing learning rate asymptotic](https://mlsanity.com/paper/2012.14657/) (2020)

## Abstract (arXiv)

We define infinitesimal gradient boosting as a limit of the popular tree-based gradient boosting algorithm from machine learning. The limit is considered in the vanishing-learning-rate asymptotic, that is when the learning rate tends to zero and the number of gradient trees is rescaled accordingly. For this purpose, we introduce a new class of randomized regression trees bridging totally randomized trees and Extra Trees and using a softmax distribution for binary splitting. Our main result is the convergence of the associated stochastic algorithm and the characterization of the limiting procedure as the unique solution of a nonlinear ordinary differential equation in a infinite dimensional function space. Infinitesimal gradient boosting defines a smooth path in the space of continuous functions along which the training error decreases, the residuals remain centered and the total variation is well controlled.