# RuleMatrix: Visualizing and Understanding Classifiers with Rules

arXiv: https://arxiv.org/abs/1807.06228 · HTML: https://mlsanity.com/paper/1807.06228/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2018.

## Built on by (most cited first, 30 of 62)

- [Benchmarking and Survey of Explanation Methods for Black Box Models](https://mlsanity.com/paper/2102.13076/) (2021)
- [Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond](https://mlsanity.com/paper/2102.01998/) (2021)
- [Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction](https://mlsanity.com/paper/2409.00265/) (2024)
- [The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations](https://mlsanity.com/paper/2212.11737/) (2022)
- [LLM Comparator: Visual Analytics for Side-by-Side Evaluation of Large Language Models](https://mlsanity.com/paper/2402.10524/) (2024)
- [Aligning Eyes between Humans and Deep Neural Network through Interactive Attention Alignment](https://mlsanity.com/paper/2202.02838/) (2022)
- [To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods](https://mlsanity.com/paper/2106.00461/) (2021)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?](https://mlsanity.com/paper/2304.04780/) (2023)
- [A Holistic Approach to Interpretability in Financial Lending: Models, Visualizations, and Summary-Explanations](https://mlsanity.com/paper/2106.02605/) (2021)
- [Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaboration](https://mlsanity.com/paper/2309.15723/) (2023)
- [VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Models](https://mlsanity.com/paper/2108.02550/) (2021)
- [Efficient Decompositional Rule Extraction for Deep Neural Networks](https://mlsanity.com/paper/2111.12628/) (2021)
- [Explainable AI for Bioinformatics: Methods, Tools, and Applications](https://mlsanity.com/paper/2212.13261/) (2022)
- [Towards Visual Explainable Active Learning for Zero-Shot Classification](https://mlsanity.com/paper/2108.06730/) (2021)
- [Learning-From-Disagreement: A Model Comparison and Visual Analytics Framework](https://mlsanity.com/paper/2201.07849/) (2022)
- [Developing a Fidelity Evaluation Approach for Interpretable Machine Learning](https://mlsanity.com/paper/2106.08492/) (2021)
- [VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees](https://mlsanity.com/paper/2112.00334/) (2021)
- [Towards an Understanding and Explanation for Mixed-Initiative Artificial Scientific Text Detection](https://mlsanity.com/paper/2304.05011/) (2023)
- [Leveraging Model-based Trees as Interpretable Surrogate Models for Model Distillation](https://mlsanity.com/paper/2310.03112/) (2023)
- [Deconstructing Human-AI Collaboration: Agency, Interaction, and Adaptation](https://mlsanity.com/paper/2404.12056/) (2024)
- [An Exploration And Validation of Visual Factors in Understanding Classification Rule Sets](https://mlsanity.com/paper/2109.09160/) (2021)
- [Visual Exploration of Machine Learning Model Behavior with Hierarchical Surrogate Rule Sets](https://mlsanity.com/paper/2201.07724/) (2022)
- [HardVis: Visual Analytics to Handle Instance Hardness Using Undersampling and Oversampling Techniques](https://mlsanity.com/paper/2203.15753/) (2022)
- [Visual Explanations with Attributions and Counterfactuals on Time Series Classification](https://mlsanity.com/paper/2307.08494/) (2023)
- [Multivariate Data Explanation by Jumping Emerging Patterns Visualization](https://mlsanity.com/paper/2106.11112/) (2021)
- [Non-linear Visual Knowledge Discovery with Elliptic Paired Coordinates](https://mlsanity.com/paper/2107.04974/) (2021)
- [Interpreting Black-box Machine Learning Models for High Dimensional Datasets](https://mlsanity.com/paper/2208.13405/) (2022)
- [Invisible Users: Uncovering End-Users' Requirements for Explainable AI via Explanation Forms and Goals](https://mlsanity.com/paper/2302.06609/) (2023)
- [MOUNTAINEER: Topology-Driven Visual Analytics for Comparing Local Explanations](https://mlsanity.com/paper/2406.15613/) (2024)

## Abstract (arXiv)

With the growing adoption of machine learning techniques, there is a surge of research interest towards making machine learning systems more transparent and interpretable. Various visualizations have been developed to help model developers understand, diagnose, and refine machine learning models. However, a large number of potential but neglected users are the domain experts with little knowledge of machine learning but are expected to work with machine learning systems. In this paper, we present an interactive visualization technique to help users with little expertise in machine learning to understand, explore and validate predictive models. By viewing the model as a black box, we extract a standardized rule-based knowledge representation from its input-output behavior. We design RuleMatrix, a matrix-based visualization of rules to help users navigate and verify the rules and the black-box model. We evaluate the effectiveness of RuleMatrix via two use cases and a usability study.