# Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP)

arXiv: https://arxiv.org/abs/1806.00663 · HTML: https://mlsanity.com/paper/1806.00663/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2018.

## Built on by (most cited first, 19 of 19)

- [SurvNAM: The machine learning survival model explanation](https://mlsanity.com/paper/2104.08903/) (2021)
- [Explainable Deep Learning in Healthcare: A Methodological Survey from an Attribution View](https://mlsanity.com/paper/2112.02625/) (2021)
- [Model Explainability in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/2106.07410/) (2021)
- [Leveraging Model-based Trees as Interpretable Surrogate Models for Model Distillation](https://mlsanity.com/paper/2310.03112/) (2023)
- [Which LIME should I trust? Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2503.24365/) (2025)
- [May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability](https://mlsanity.com/paper/2309.13965/) (2023)
- [Bias, Fairness, and Accountability with AI and ML Algorithms](https://mlsanity.com/paper/2105.06558/) (2021)
- [SurvBeX: An explanation method of the machine learning survival models based on the Beran estimator](https://mlsanity.com/paper/2308.03730/) (2023)
- [LIMEcraft: Handcrafted superpixel selection and inspection for Visual eXplanations](https://mlsanity.com/paper/2111.08094/) (2021)
- [Approximation of group explainers with coalition structure using Monte Carlo sampling on the product space of coalitions and features](https://mlsanity.com/paper/2303.10216/) (2023)
- [Conversational Explanations: Discussing Explainable AI with Non-AI Experts](https://mlsanity.com/paper/2503.16444/) (2025)
- [Model interpretation using improved local regression with variable importance](https://mlsanity.com/paper/2209.05371/) (2022)
- [Causality-Aware Local Interpretable Model-Agnostic Explanations](https://mlsanity.com/paper/2212.05256/) (2022)
- [Multi-resolution Interpretation and Diagnostics Tool for Natural Language Classifiers](https://mlsanity.com/paper/2303.03542/) (2023)
- [SurvBeNIM: The Beran-Based Neural Importance Model for Explaining the Survival Models](https://mlsanity.com/paper/2312.06638/) (2023)
- [Comparison of decision trees with Local Interpretable Model-Agnostic Explanations (LIME) technique and multi-linear regression for explaining support vector regression model in terms of root mean square error (RMSE) values](https://mlsanity.com/paper/2404.07046/) (2024)
- [MindfulLIME: A Stable Solution for Explanations of Machine Learning Models with Enhanced Localization Precision -- A Medical Image Case Study](https://mlsanity.com/paper/2503.20758/) (2025)
- [Explainable post-training bias mitigation with distribution-based fairness metrics](https://mlsanity.com/paper/2504.01223/) (2025)
- [Beyond Linear Surrogates: High-Fidelity Local Explanations for Black-Box Models](https://mlsanity.com/paper/2512.05556/) (2025)

## Abstract (arXiv)

Supervised Machine Learning (SML) algorithms such as Gradient Boosting, Random Forest, and Neural Networks have become popular in recent years due to their increased predictive performance over traditional statistical methods. This is especially true with large data sets (millions or more observations and hundreds to thousands of predictors). However, the complexity of the SML models makes them opaque and hard to interpret without additional tools. There has been a lot of interest recently in developing global and local diagnostics for interpreting and explaining SML models. In this paper, we propose locally interpretable models and effects based on supervised partitioning (trees) referred to as LIME-SUP. This is in contrast with the KLIME approach that is based on clustering the predictor space. We describe LIME-SUP based on fitting trees to the fitted response (LIM-SUP-R) as well as the derivatives of the fitted response (LIME-SUP-D). We compare the results with KLIME and describe its advantages using simulation and real data.