# Gradient boosting with vector-valued leafs

arXiv: https://arxiv.org/abs/2606.29326 · HTML: https://mlsanity.com/paper/2606.29326/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 5 references that are in the map.
- First posted on arXiv 28 Jun 2026.

## Builds on (most cited first, 3 of 3)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [CatBoost: gradient boosting with categorical features support](https://mlsanity.com/paper/1810.11363/) (2018)
- [A Blockwise Descent Algorithm for Group-penalized Multiresponse and Multinomial Regression](https://mlsanity.com/paper/1311.6529/) (2013)

## Abstract (arXiv)

Gradient boosting in the form of decision tree ensembles has successfully been applied to a variety of problems using simple objective functions based on log-likelihoods of a single variable. The concept extends naturally to objective functions operating on vectors - for example, multinomial logistic log-likelihood for multi-class classification, where observations have a score for each class - but popular frameworks approach these functions by either updating one value of the input vectors at a time, or by using a diagonal upper bound on the second derivative. This work extends the usual gradient boosting framework to functions of vector inputs and sketches a simple algorithm that can be used efficiently with histogram-based decision trees.