TRBoost: A Generic Gradient Boosting Machine based on Trust-region Method
Not on the map. Why?
Abstract (arXiv)
Gradient Boosting Machines (GBMs) have demonstrated remarkable success in solving diverse problems by utilizing Taylor expansions in functional space. However, achieving a balance between performance and generality has posed a challenge for GBMs. In particular, gradient descent-based GBMs employ the first-order Taylor expansion to ensure applicability to all loss functions, while Newton's method-based GBMs use positive Hessian information to achieve superior performance at the expense of generality. To address this issue, this study proposes a new generic Gradient Boosting Machine called Trust-region Boosting (TRBoost). In each iteration, TRBoost uses a constrained quadratic model to approximate the objective and applies the Trust-region algorithm to solve it and obtain a new learner. Unlike Newton's method-based GBMs, TRBoost does not require the Hessian to be positive definite, thereby allowing it to be applied to arbitrary loss functions while still maintaining competitive performance similar to second-order algorithms. The convergence analysis and numerical experiments conducted in this study confirm that TRBoost is as general as first-order GBMs and yields competitive results compared to second-order GBMs. Overall, TRBoost is a promising approach that balances performance and generality, making it a valuable addition to the toolkit of machine learning practitioners.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 11 of its 42 references that are in the map. First posted on arXiv 28 Sep 2022.
Builds on
11 of 42 references.
References · 11
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
- 2014OpenML: networked science in machine learningcited by 523
- 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
- 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
- 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
- 2019A Survey on Distributed Machine Learningcited by 187
- 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
- 2019NGBoost: Natural Gradient Boosting for Probabilistic Predictioncited by 94
- 2018Gradient and Newton Boosting for Classification and Regressioncited by 14
- 2018Accelerated Gradient Boostingcited by 7
Built on by
Citing papers per year: 2023: 2; 2024: 1.
3 papers.
- 2023Robust-GBDT: GBDT with Nonconvex Loss for Tabular Classification in the Presence of Label Noise and Class Imbalancecited by 0
- 2023Machine Learning For An Explainable Cost Prediction of Medical Insurancecited by 0
- 2024Improving GBDT Performance on Imbalanced Datasets: An Empirical Study of Class-Balanced Loss Functionscited by 0
- 2024Improving GBDT Performance on Imbalanced Datasets: An Empirical Study of Class-Balanced Loss Functions19 Jul 2024
- 2023Machine Learning For An Explainable Cost Prediction of Medical Insurance23 Nov 2023
- 2023Robust-GBDT: GBDT with Nonconvex Loss for Tabular Classification in the Presence of Label Noise and Class Imbalance8 Oct 2023