# Regularizing Black-box Models for Improved Interpretability

arXiv: https://arxiv.org/abs/1902.06787 · HTML: https://mlsanity.com/paper/1902.06787/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2019.

## Built on by (most cited first, 30 of 35)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond](https://mlsanity.com/paper/2103.10689/) (2021)
- [Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations](https://mlsanity.com/paper/2210.11584/) (2022)
- [A Survey on Interpretable Reinforcement Learning](https://mlsanity.com/paper/2112.13112/) (2021)
- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [Concept-level Debugging of Part-Prototype Networks](https://mlsanity.com/paper/2205.15769/) (2022)
- [Shapley Explanation Networks](https://mlsanity.com/paper/2104.02297/) (2021)
- [DISSECT: Disentangled Simultaneous Explanations via Concept Traversals](https://mlsanity.com/paper/2105.15164/) (2021)
- [Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning](https://mlsanity.com/paper/2212.03954/) (2022)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Refining Language Models with Compositional Explanations](https://mlsanity.com/paper/2103.10415/) (2021)
- [The Role of Explainability in Assuring Safety of Machine Learning in Healthcare](https://mlsanity.com/paper/2109.00520/) (2021)
- [What to Learn, and How: Toward Effective Learning from Rationales](https://mlsanity.com/paper/2112.00071/) (2021)
- [Self-Interpretable Time Series Prediction with Counterfactual Explanations](https://mlsanity.com/paper/2306.06024/) (2023)
- [Use-Case-Grounded Simulations for Explanation Evaluation](https://mlsanity.com/paper/2206.02256/) (2022)
- [Interpreting Unfairness in Graph Neural Networks via Training Node Attribution](https://mlsanity.com/paper/2211.14383/) (2022)
- [FINER: Enhancing State-of-the-art Classifiers with Feature Attribution to Facilitate Security Analysis](https://mlsanity.com/paper/2308.05362/) (2023)
- [Learning to Predict with Supporting Evidence: Applications to Clinical Risk Prediction](https://mlsanity.com/paper/2103.02768/) (2021)
- [Perspectives on Incorporating Expert Feedback into Model Updates](https://mlsanity.com/paper/2205.06905/) (2022)
- [Defense Against Explanation Manipulation](https://mlsanity.com/paper/2111.04303/) (2021)
- [Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales](https://mlsanity.com/paper/2404.03098/) (2024)
- [Interpretable Alzheimer's Disease Classification Via a Contrastive Diffusion Autoencoder](https://mlsanity.com/paper/2306.03022/) (2023)
- [Interpretability in Symbolic Regression: a benchmark of Explanatory Methods using the Feynman data set](https://mlsanity.com/paper/2404.05908/) (2024)
- [Responsibility: An Example-based Explainable AI approach via Training Process Inspection](https://mlsanity.com/paper/2209.03433/) (2022)
- [Joint Explainability-Performance Optimization With Surrogate Models for AI-Driven Edge Services](https://mlsanity.com/paper/2503.07784/) (2025)
- [Training Deep Models to be Explained with Fewer Examples](https://mlsanity.com/paper/2112.03508/) (2021)
- [Visualizing Transferred Knowledge: An Interpretive Model of Unsupervised Domain Adaptation](https://mlsanity.com/paper/2303.02302/) (2023)
- [Explanation Regularisation through the Lens of Attributions](https://mlsanity.com/paper/2407.16693/) (2024)
- [Harnessing value from data science in business: ensuring explainability and fairness of solutions](https://mlsanity.com/paper/2108.07714/) (2021)

## Abstract (arXiv)

Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, whose explanation quality can be unpredictable. Our method, ExpO, is a hybridization of these approaches that regularizes a model for explanation quality at training time. Importantly, these regularizers are differentiable, model agnostic, and require no domain knowledge to define. We demonstrate that post-hoc explanations for ExpO-regularized models have better explanation quality, as measured by the common fidelity and stability metrics. We verify that improving these metrics leads to significantly more useful explanations with a user study on a realistic task.