# Using Model-Based Trees with Boosting to Fit Low-Order Functional ANOVA Models

arXiv: https://arxiv.org/abs/2207.06950 · HTML: https://mlsanity.com/paper/2207.06950/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 14 Jul 2022.

## Builds on (most cited first, 8 of 8)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions](https://mlsanity.com/paper/2003.07132/) (2020)
- [Purifying Interaction Effects with the Functional ANOVA: An Efficient Algorithm for Recovering Identifiable Additive Models](https://mlsanity.com/paper/1911.04974/) (2019)
- [Explainable Neural Networks based on Additive Index Models](https://mlsanity.com/paper/1806.01933/) (2018)
- [Unwrapping The Black Box of Deep ReLU Networks: Interpretability, Diagnostics, and Simplification](https://mlsanity.com/paper/2011.04041/) (2020)
- [Surrogate Locally-Interpretable Models with Supervised Machine Learning Algorithms](https://mlsanity.com/paper/2007.14528/) (2020)

## Built on by (most cited first, 3 of 3)

- [Interpretable Machine Learning based on Functional ANOVA Framework: Algorithms and Comparisons](https://mlsanity.com/paper/2305.15670/) (2023)
- [Monotone Tree-Based GAMI Models by Adapting XGBoost](https://mlsanity.com/paper/2309.02426/) (2023)
- [Computing SHAP Efficiently Using Model Structure Information](https://mlsanity.com/paper/2309.02417/) (2023)

## Abstract (arXiv)

Low-order functional ANOVA (fANOVA) models have been rediscovered in the machine learning (ML) community under the guise of inherently interpretable machine learning. Explainable Boosting Machines or EBM (Lou et al. 2013) and GAMI-Net (Yang et al. 2021) are two recently proposed ML algorithms for fitting functional main effects and second-order interactions. We propose a new algorithm, called GAMI-Tree, that is similar to EBM, but has a number of features that lead to better performance. It uses model-based trees as base learners and incorporates a new interaction filtering method that is better at capturing the underlying interactions. In addition, our iterative training method converges to a model with better predictive performance, and the embedded purification ensures that interactions are hierarchically orthogonal to main effects. The algorithm does not need extensive tuning, and our implementation is fast and efficient. We use simulated and real datasets to compare the performance and interpretability of GAMI-Tree with EBM and GAMI-Net.