# Soft Gradient Boosting Machine

arXiv: https://arxiv.org/abs/2006.04059 · HTML: https://mlsanity.com/paper/2006.04059/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2020.

## Built on by (most cited first, 7 of 7)

- [AGBoost: Attention-based Modification of Gradient Boosting Machine](https://mlsanity.com/paper/2207.05724/) (2022)
- [A Hybrid Framework for Sequential Data Prediction with End-to-End Optimization](https://mlsanity.com/paper/2203.13787/) (2022)
- [Learning Multi-Layered GBDT Via Back Propagation](https://mlsanity.com/paper/2109.11863/) (2021)
- [Variational Boosted Soft Trees](https://mlsanity.com/paper/2302.10706/) (2023)
- [Self-Error Adjustment: Theory and Practice of Balancing Individual Performance and Diversity in Ensemble Learning](https://mlsanity.com/paper/2508.04948/) (2025)
- [Soft Gradient Boosting with Learnable Feature Transforms for Sequential Regression](https://mlsanity.com/paper/2509.12920/) (2025)
- [PolyMon: A Unified Framework for Polymer Property Prediction](https://mlsanity.com/paper/2603.13303/) (2026)

## Abstract (arXiv)

Gradient Boosting Machine has proven to be one successful function approximator and has been widely used in a variety of areas. However, since the training procedure of each base learner has to take the sequential order, it is infeasible to parallelize the training process among base learners for speed-up. In addition, under online or incremental learning settings, GBMs achieved sub-optimal performance due to the fact that the previously trained base learners can not adapt with the environment once trained. In this work, we propose the soft Gradient Boosting Machine (sGBM) by wiring multiple differentiable base learners together, by injecting both local and global objectives inspired from gradient boosting, all base learners can then be jointly optimized with linear speed-up. When using differentiable soft decision trees as base learner, such device can be regarded as an alternative version of the (hard) gradient boosting decision trees with extra benefits. Experimental results showed that, sGBM enjoys much higher time efficiency with better accuracy, given the same base learner in both on-line and off-line settings.