# Enabling Regional Explainability by Automatic and Model-agnostic Rule Extraction

arXiv: https://arxiv.org/abs/2406.17885 · HTML: https://mlsanity.com/paper/2406.17885/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 59 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Understanding Neural Networks Through Deep Visualization](https://mlsanity.com/paper/1506.06579/) (2015)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Distilling a Neural Network Into a Soft Decision Tree](https://mlsanity.com/paper/1711.09784/) (2017)
- [Interpretable Convolutional Neural Networks](https://mlsanity.com/paper/1710.00935/) (2017)
- [From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation](https://mlsanity.com/paper/2206.03208/) (2022)
- [Scalable Bayesian Rule Lists](https://mlsanity.com/paper/1602.08610/) (2016)
- [Falling Rule Lists](https://mlsanity.com/paper/1411.5899/) (2014)
- [A Disentangling Invertible Interpretation Network for Explaining Latent Representations](https://mlsanity.com/paper/2004.13166/) (2020)
- [Explaining Latent Representations with a Corpus of Examples](https://mlsanity.com/paper/2110.15355/) (2021)
- [Rule Extraction Algorithm for Deep Neural Networks: A Review](https://mlsanity.com/paper/1610.05267/) (2016)
- [Regional Tree Regularization for Interpretability in Black Box Models](https://mlsanity.com/paper/1908.04494/) (2019)
- [Automatically Explaining Machine Learning Prediction Results: A Demonstration on Type 2 Diabetes Risk Prediction](https://mlsanity.com/paper/1812.02852/) (2018)
- [Discovering Categorical Main and Interaction Effects Based on Association Rule Mining](https://mlsanity.com/paper/2104.04728/) (2021)
- [NN2Rules: Extracting Rule List from Neural Networks](https://mlsanity.com/paper/2207.12271/) (2022)

## Abstract (arXiv)

In Explainable AI, rule extraction translates model knowledge into logical rules, such as IF-THEN statements, crucial for understanding patterns learned by black-box models. This could significantly aid in fields like disease diagnosis, disease progression estimation, or drug discovery. However, such application domains often contain imbalanced data, with the class of interest underrepresented. Existing methods inevitably compromise the performance of rules for the minor class to maximise the overall performance. As the first attempt in this field, we propose a model-agnostic approach for extracting rules from specific subgroups of data, featuring automatic rule generation for numerical features. This method enhances the regional explainability of machine learning models and offers wider applicability compared to existing methods. We additionally introduce a new method for selecting features to compose rules, reducing computational costs in high-dimensional spaces. Experiments across various datasets and models demonstrate the effectiveness of our methods.