# Distributional Adaptive Soft Regression Trees

arXiv: https://arxiv.org/abs/2210.10389 · HTML: https://mlsanity.com/paper/2210.10389/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 47 references that are in the map.
- First posted on arXiv 19 Oct 2022.

## Builds on (most cited first, 6 of 6)

- [Distilling a Neural Network Into a Soft Decision Tree](https://mlsanity.com/paper/1711.09784/) (2017)
- [Bayesian Regression Tree Ensembles that Adapt to Smoothness and Sparsity](https://mlsanity.com/paper/1707.09461/) (2017)
- [To tune or not to tune the number of trees in random forest?](https://mlsanity.com/paper/1705.05654/) (2017)
- [Distributional Regression Forests for Probabilistic Precipitation Forecasting in Complex Terrain](https://mlsanity.com/paper/1804.02921/) (2018)
- [M5 Competition Uncertainty: Overdispersion, distributional forecasting, GAMLSS and beyond](https://mlsanity.com/paper/2107.06675/) (2021)
- [A comparative study of non-deep learning, deep learning, and ensemble learning methods for sunspot number prediction](https://mlsanity.com/paper/2203.05757/) (2022)

## Abstract (arXiv)

Random forests are an ensemble method relevant for many problems, such as regression or classification. They are popular due to their good predictive performance (compared to, e.g., decision trees) requiring only minimal tuning of hyperparameters. They are built via aggregation of multiple regression trees during training and are usually calculated recursively using hard splitting rules. Recently regression forests have been incorporated into the framework of distributional regression, a nowadays popular regression approach aiming at estimating complete conditional distributions rather than relating the mean of an output variable to input features only - as done classically. This article proposes a new type of a distributional regression tree using a multivariate soft split rule. One great advantage of the soft split is that smooth high-dimensional functions can be estimated with only one tree while the complexity of the function is controlled adaptive by information criteria. Moreover, the search for the optimal split variable is obsolete. We show by means of extensive simulation studies that the algorithm has excellent properties and outperforms various benchmark methods, especially in the presence of complex non-linear feature interactions. Finally, we illustrate the usefulness of our approach with an example on probabilistic forecasts for the Sun's activity.