# A novel gradient-based method for decision trees optimizing arbitrary differential loss functions

arXiv: https://arxiv.org/abs/2503.17855 · HTML: https://mlsanity.com/paper/2503.17855/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 32 references that are in the map.
- First posted on arXiv 22 Mar 2025.

## Builds on (most cited first, 8 of 8)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Machine Learning for Survival Analysis: A Survey](https://mlsanity.com/paper/1708.04649/) (2017)
- [Optimal Sparse Decision Trees](https://mlsanity.com/paper/1904.12847/) (2019)
- [Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles](https://mlsanity.com/paper/2401.12069/) (2024)
- [An Introduction to Deep Survival Analysis Models for Predicting Time-to-Event Outcomes](https://mlsanity.com/paper/2410.01086/) (2024)
- [HHCART: An Oblique Decision Tree](https://mlsanity.com/paper/1504.03415/) (2015)
- [Optimal Survival Trees](https://mlsanity.com/paper/2012.04284/) (2020)
- [Interpretable Ensembles of Hyper-Rectangles as Base Models](https://mlsanity.com/paper/2303.08625/) (2023)

## Abstract (arXiv)

There are many approaches for training decision trees. This work introduces a novel gradient-based method for constructing decision trees that optimize arbitrary differentiable loss functions, overcoming the limitations of heuristic splitting rules. Unlike traditional approaches that rely on heuristic splitting rules, the proposed method refines predictions using the first and second derivatives of the loss function, enabling the optimization of complex tasks such as classification, regression, and survival analysis. We demonstrate the method's applicability to classification, regression, and survival analysis tasks, including those with censored data. Numerical experiments on both real and synthetic datasets compare the proposed method with traditional decision tree algorithms, such as CART, Extremely Randomized Trees, and SurvTree. The implementation of the method is publicly available, providing a practical tool for researchers and practitioners. This work advances the field of decision tree-based modeling, offering a more flexible and accurate approach for handling structured data and complex tasks. By leveraging gradient-based optimization, the proposed method bridges the gap between traditional decision trees and modern machine learning techniques, paving the way for further innovations in interpretable and high-performing models.