# RuleExplorer: A Scalable Matrix Visualization for Understanding Tree Ensemble Classifiers

arXiv: https://arxiv.org/abs/2409.03164 · HTML: https://mlsanity.com/paper/2409.03164/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 76 references that are in the map.
- First posted on arXiv 5 Sep 2024.

## Builds on (most cited first, 30 of 31)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Techniques for Interpretable Machine Learning](https://mlsanity.com/paper/1808.00033/) (2018)
- [Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models](https://mlsanity.com/paper/1804.09299/) (2018)
- [Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation](https://mlsanity.com/paper/1710.06169/) (2017)
- [RuleMatrix: Visualizing and Understanding Classifiers with Rules](https://mlsanity.com/paper/1807.06228/) (2018)
- [Do Convolutional Neural Networks Learn Class Hierarchy?](https://mlsanity.com/paper/1710.06501/) (2017)
- [RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records](https://mlsanity.com/paper/1805.10724/) (2018)
- [Towards Better Analysis of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1604.07043/) (2016)
- [Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models](https://mlsanity.com/paper/2009.07053/) (2020)
- [Deep Forest](https://mlsanity.com/paper/1702.08835/) (2017)
- [Understanding Hidden Memories of Recurrent Neural Networks](https://mlsanity.com/paper/1710.10777/) (2017)
- [Born-Again Tree Ensembles](https://mlsanity.com/paper/2003.11132/) (2020)
- [OoDAnalyzer: Interactive Analysis of Out-of-Distribution Samples](https://mlsanity.com/paper/2002.03103/) (2020)
- [Foundation Models Meet Visualizations: Challenges and Opportunities](https://mlsanity.com/paper/2310.05771/) (2023)
- [Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach](https://mlsanity.com/paper/1606.09066/) (2016)
- [Explainable Matrix -- Visualization for Global and Local Interpretability of Random Forest Classification Ensembles](https://mlsanity.com/paper/2005.04289/) (2020)
- [Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels](https://mlsanity.com/paper/2110.12536/) (2021)
- [Interactive Steering of Hierarchical Clustering](https://mlsanity.com/paper/2009.09618/) (2020)
- [Diagnosing Ensemble Few-Shot Classifiers](https://mlsanity.com/paper/2206.04372/) (2022)
- [TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualization](https://mlsanity.com/paper/2209.09227/) (2022)
- [VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees](https://mlsanity.com/paper/2112.00334/) (2021)
- [A Unified Interactive Model Evaluation for Classification, Object Detection, and Instance Segmentation in Computer Vision](https://mlsanity.com/paper/2308.05168/) (2023)
- [RNNbow: Visualizing Learning via Backpropagation Gradients in Recurrent Neural Networks](https://mlsanity.com/paper/1907.12545/) (2019)
- [Cluster-Aware Grid Layout](https://mlsanity.com/paper/2308.03651/) (2023)
- [An Exploration And Validation of Visual Factors in Understanding Classification Rule Sets](https://mlsanity.com/paper/2109.09160/) (2021)
- [Visual Exploration of Machine Learning Model Behavior with Hierarchical Surrogate Rule Sets](https://mlsanity.com/paper/2201.07724/) (2022)
- [Enhancing Single-Frame Supervision for Better Temporal Action Localization](https://mlsanity.com/paper/2312.05178/) (2023)
- [Visualizing RNN States with Predictive Semantic Encodings](https://mlsanity.com/paper/1908.00588/) (2019)
- [Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles](https://mlsanity.com/paper/2208.09665/) (2022)
- [DeforestVis: Behavior Analysis of Machine Learning Models with Surrogate Decision Stumps](https://mlsanity.com/paper/2304.00133/) (2023)

## Built on by (most cited first, 1 of 1)

- [WidgetVA: A Widget-Centric Framework and Benchmark for Agentic Visual Analytics](https://mlsanity.com/paper/2609.24094/) (2026)

## Abstract (arXiv)

The high performance of tree ensemble classifiers benefits from a large set of rules, which, in turn, makes the models hard to understand. To improve interpretability, existing methods extract a subset of rules for approximation using model reduction techniques. However, by focusing on the reduced rule set, these methods often lose fidelity and ignore anomalous rules that, despite their infrequency, play crucial roles in real-world applications. This paper introduces a scalable visual analysis method to explain tree ensemble classifiers that contain tens of thousands of rules. The key idea is to address the issue of losing fidelity by adaptively organizing the rules as a hierarchy rather than reducing them. To ensure the inclusion of anomalous rules, we develop an anomaly-biased model reduction method to prioritize these rules at each hierarchical level. Synergized with this hierarchical organization of rules, we develop a matrix-based hierarchical visualization to support exploration at different levels of detail. Our quantitative experiments and case studies demonstrate how our method fosters a deeper understanding of both common and anomalous rules, thereby enhancing interpretability without sacrificing comprehensiveness.