# Fast Linear Model Trees by PILOT

arXiv: https://arxiv.org/abs/2302.03931 · HTML: https://mlsanity.com/paper/2302.03931/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 64 references that are in the map.
- First posted on arXiv 8 Feb 2023.

## Builds on (most cited first, 8 of 8)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [An Introduction to Matrix Concentration Inequalities](https://mlsanity.com/paper/1501.01571/) (2015)
- [Consistency of random forests](https://mlsanity.com/paper/1405.2881/) (2014)
- [High-Dimensional Inference: Confidence Intervals, p-Values and R-Software hdi](https://mlsanity.com/paper/1408.4026/) (2014)
- [Convergence Rates of Oblique Regression Trees for Flexible Function Libraries](https://mlsanity.com/paper/2210.14429/) (2022)
- [Gradient Boosting With Piece-Wise Linear Regression Trees](https://mlsanity.com/paper/1802.05640/) (2018)
- [A New Perspective on Boosting in Linear Regression via Subgradient Optimization and Relatives](https://mlsanity.com/paper/1505.04243/) (2015)
- [A comparative study of forecasting Corporate Credit Ratings using Neural Networks, Support Vector Machines, and Decision Trees](https://mlsanity.com/paper/2007.06617/) (2020)

## Built on by (most cited first, 7 of 7)

- [Dividable Configuration Performance Learning](https://mlsanity.com/paper/2409.07629/) (2024)
- [Tessellated Linear Model for Age Prediction from Voice](https://mlsanity.com/paper/2501.09229/) (2025)
- [Experiments with Optimal Model Trees](https://mlsanity.com/paper/2503.12902/) (2025)
- [Towards Accurate Forecasting of Renewable Energy : Building Datasets and Benchmarking Machine Learning Models for Solar and Wind Power in France](https://mlsanity.com/paper/2504.16100/) (2025)
- [Convolutional Model Trees](https://mlsanity.com/paper/2511.12725/) (2025)
- [SplitWise Regression: Stepwise Modeling with Adaptive Dummy Encoding](https://mlsanity.com/paper/2505.15423/) (2025)
- [Uncertainty-Aware Extrapolation in Bayesian Oblique Trees](https://mlsanity.com/paper/2601.22899/) (2026)

## Abstract (arXiv)

Linear model trees are regression trees that incorporate linear models in the leaf nodes. This preserves the intuitive interpretation of decision trees and at the same time enables them to better capture linear relationships, which is hard for standard decision trees. But most existing methods for fitting linear model trees are time consuming and therefore not scalable to large data sets. In addition, they are more prone to overfitting and extrapolation issues than standard regression trees. In this paper we introduce PILOT, a new algorithm for linear model trees that is fast, regularized, stable and interpretable. PILOT trains in a greedy fashion like classic regression trees, but incorporates an L2 boosting approach and a model selection rule for fitting linear models in the nodes. The abbreviation PILOT stands for PIecewise Linear Organic Tree, where `organic' refers to the fact that no pruning is carried out. PILOT has the same low time and space complexity as CART without its pruning. An empirical study indicates that PILOT tends to outperform standard decision trees and other linear model trees on a variety of data sets. Moreover, we prove its consistency in an additive model setting under weak assumptions. When the data is generated by a linear model, the convergence rate is polynomial.