# DeforestVis: Behavior Analysis of Machine Learning Models with Surrogate Decision Stumps

arXiv: https://arxiv.org/abs/2304.00133 · HTML: https://mlsanity.com/paper/2304.00133/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 105 references that are in the map.
- First posted on arXiv 31 Mar 2023.

## Builds on (most cited first, 25 of 25)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers](https://mlsanity.com/paper/1801.06889/) (2018)
- [Interpreting CNNs via Decision Trees](https://mlsanity.com/paper/1802.00121/) (2018)
- [RuleMatrix: Visualizing and Understanding Classifiers with Rules](https://mlsanity.com/paper/1807.06228/) (2018)
- [t-viSNE: Interactive Assessment and Interpretation of t-SNE Projections](https://mlsanity.com/paper/2002.06910/) (2020)
- [The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations](https://mlsanity.com/paper/2212.11737/) (2022)
- [Clustrophile 2: Guided Visual Clustering Analysis](https://mlsanity.com/paper/1804.03048/) (2018)
- [Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values](https://mlsanity.com/paper/2206.15465/) (2022)
- [Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics](https://mlsanity.com/paper/1907.07296/) (2019)
- [Explainable Matrix -- Visualization for Global and Local Interpretability of Random Forest Classification Ensembles](https://mlsanity.com/paper/2005.04289/) (2020)
- [StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics](https://mlsanity.com/paper/2005.01575/) (2020)
- [Evaluation of Sampling Methods for Scatterplots](https://mlsanity.com/paper/2007.14666/) (2020)
- [VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees](https://mlsanity.com/paper/2112.00334/) (2021)
- [FeatureEnVi: Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches](https://mlsanity.com/paper/2103.14539/) (2021)
- [An Exploration And Validation of Visual Factors in Understanding Classification Rule Sets](https://mlsanity.com/paper/2109.09160/) (2021)
- [Visual Exploration of Machine Learning Model Behavior with Hierarchical Surrogate Rule Sets](https://mlsanity.com/paper/2201.07724/) (2022)
- [HardVis: Visual Analytics to Handle Instance Hardness Using Undersampling and Oversampling Techniques](https://mlsanity.com/paper/2203.15753/) (2022)
- [Multivariate Data Explanation by Jumping Emerging Patterns Visualization](https://mlsanity.com/paper/2106.11112/) (2021)
- [An Interactive Machine Learning Framework](https://mlsanity.com/paper/1610.05463/) (2016)
- [ExplorerTree: a focus+context exploration approach for 2D embeddings](https://mlsanity.com/paper/2106.10592/) (2021)

## Built on by (most cited first, 2 of 2)

- [Visual Analytics for Explainable and Trustworthy Artificial Intelligence](https://mlsanity.com/paper/2507.10240/) (2025)
- [RuleExplorer: A Scalable Matrix Visualization for Understanding Tree Ensemble Classifiers](https://mlsanity.com/paper/2409.03164/) (2024)

## Abstract (arXiv)

As the complexity of machine learning (ML) models increases and their application in different (and critical) domains grows, there is a strong demand for more interpretable and trustworthy ML. A direct, model-agnostic, way to interpret such models is to train surrogate models-such as rule sets and decision trees-that sufficiently approximate the original ones while being simpler and easier-to-explain. Yet, rule sets can become very lengthy, with many if-else statements, and decision tree depth grows rapidly when accurately emulating complex ML models. In such cases, both approaches can fail to meet their core goal-providing users with model interpretability. To tackle this, we propose DeforestVis, a visual analytics tool that offers summarization of the behaviour of complex ML models by providing surrogate decision stumps (one-level decision trees) generated with the Adaptive Boosting (AdaBoost) technique. DeforestVis helps users to explore the complexity versus fidelity trade-off by incrementally generating more stumps, creating attribute-based explanations with weighted stumps to justify decision making, and analysing the impact of rule overriding on training instance allocation between one or more stumps. An independent test set allows users to monitor the effectiveness of manual rule changes and form hypotheses based on case-by-case analyses. We show the applicability and usefulness of DeforestVis with two use cases and expert interviews with data analysts and model developers.