# Leveraging Model-based Trees as Interpretable Surrogate Models for Model Distillation

arXiv: https://arxiv.org/abs/2310.03112 · HTML: https://mlsanity.com/paper/2310.03112/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 14 of 14)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Distilling a Neural Network Into a Soft Decision Tree](https://mlsanity.com/paper/1711.09784/) (2017)
- [RuleMatrix: Visualizing and Understanding Classifiers with Rules](https://mlsanity.com/paper/1807.06228/) (2018)
- [S-LIME: Stabilized-LIME for Model Explanation](https://mlsanity.com/paper/2106.07875/) (2021)
- [Interpreting Blackbox Models via Model Extraction](https://mlsanity.com/paper/1705.08504/) (2017)
- [The Impact of Random Models on Clustering Similarity](https://mlsanity.com/paper/1701.06508/) (2017)
- [Defining Locality for Surrogates in Post-hoc Interpretablity](https://mlsanity.com/paper/1806.07498/) (2018)
- [OptiLIME: Optimized LIME Explanations for Diagnostic Computer Algorithms](https://mlsanity.com/paper/2006.05714/) (2020)
- [Model-based Recursive Partitioning for Subgroup Analyses](https://mlsanity.com/paper/1605.04078/) (2016)
- [Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP)](https://mlsanity.com/paper/1806.00663/) (2018)
- [REPID: Regional Effect Plots with implicit Interaction Detection](https://mlsanity.com/paper/2202.07254/) (2022)
- [Surrogate Locally-Interpretable Models with Supervised Machine Learning Algorithms](https://mlsanity.com/paper/2007.14528/) (2020)
- [Fitting Prediction Rule Ensembles with R Package pre](https://mlsanity.com/paper/1707.07149/) (2017)
- [The Power of Unbiased Recursive Partitioning: A Unifying View of CTree, MOB, and GUIDE](https://mlsanity.com/paper/1906.10179/) (2019)

## Built on by (most cited first, 8 of 8)

- [Neuron-Anchored Rule Extraction for Large Language Models via Contrastive Hierarchical Ablation](https://mlsanity.com/paper/2605.03058/) (2026)
- [Experiments with Optimal Model Trees](https://mlsanity.com/paper/2503.12902/) (2025)
- [Explanations Go Linear: Post-hoc Explainability for Tabular Data with Interpretable Meta-Encoding](https://mlsanity.com/paper/2504.20667/) (2025)
- [Transparent Adaptive Learning via Data-Centric Multimodal Explainable AI](https://mlsanity.com/paper/2508.00665/) (2025)
- [From Actions to Words: Towards Abstractive-Textual Policy Summarization in RL](https://mlsanity.com/paper/2503.10509/) (2025)
- [Interpretability-by-Design with Accurate Locally Additive Models and Conditional Feature Effects](https://mlsanity.com/paper/2602.16503/) (2026)
- [UNAD+: An Explainable Hybrid Framework for Unknown Network Attack Detection](https://mlsanity.com/paper/2605.22621/) (2026)
- [SAILS: Surrogate-based Analysis of Interactions via Local Effect Smooths](https://mlsanity.com/paper/2606.09404/) (2026)

## Abstract (arXiv)

Surrogate models play a crucial role in retrospectively interpreting complex and powerful black box machine learning models via model distillation. This paper focuses on using model-based trees as surrogate models which partition the feature space into interpretable regions via decision rules. Within each region, interpretable models based on additive main effects are used to approximate the behavior of the black box model, striking for an optimal balance between interpretability and performance. Four model-based tree algorithms, namely SLIM, GUIDE, MOB, and CTree, are compared regarding their ability to generate such surrogate models. We investigate fidelity, interpretability, stability, and the algorithms' capability to capture interaction effects through appropriate splits. Based on our comprehensive analyses, we finally provide an overview of user-specific recommendations.