# Enforcing Interpretability and its Statistical Impacts: Trade-offs between Accuracy and Interpretability

arXiv: https://arxiv.org/abs/2010.13764 · HTML: https://mlsanity.com/paper/2010.13764/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2020.

## Built on by (most cited first, 25 of 25)

- [Neurosymbolic AI for Reasoning over Knowledge Graphs: A Survey](https://mlsanity.com/paper/2302.07200/) (2023)
- [Towards Interpretable Hate Speech Detection using Large Language Model-extracted Rationales](https://mlsanity.com/paper/2403.12403/) (2024)
- [Learning Optimal Fair Classification Trees: Trade-offs Between Interpretability, Fairness, and Accuracy](https://mlsanity.com/paper/2201.09932/) (2022)
- [Feature Perturbation Augmentation for Reliable Evaluation of Importance Estimators in Neural Networks](https://mlsanity.com/paper/2303.01538/) (2023)
- [Rethinking Explainable Machine Learning as Applied Statistics](https://mlsanity.com/paper/2402.02870/) (2024)
- [Perspectives on Incorporating Expert Feedback into Model Updates](https://mlsanity.com/paper/2205.06905/) (2022)
- [A Theory of Interpretable Approximations](https://mlsanity.com/paper/2406.10529/) (2024)
- [Opportunities and limitations of explaining quantum machine learning](https://mlsanity.com/paper/2412.14753/) (2024)
- [Recent Advances in Malware Detection: Graph Learning and Explainability](https://mlsanity.com/paper/2502.10556/) (2025)
- [Neurosymbolic AI for Reasoning on Biomedical Knowledge Graphs](https://mlsanity.com/paper/2307.08411/) (2023)
- [SoK: Taming the Triangle -- On the Interplays between Fairness, Interpretability and Privacy in Machine Learning](https://mlsanity.com/paper/2312.16191/) (2023)
- [Fairness and Sparsity within Rashomon sets: Enumeration-Free Exploration and Characterization](https://mlsanity.com/paper/2502.05286/) (2025)
- [Integration of Radiomics and Tumor Biomarkers in Interpretable Machine Learning Models](https://mlsanity.com/paper/2303.11177/) (2023)
- [DISCRET: Synthesizing Faithful Explanations For Treatment Effect Estimation](https://mlsanity.com/paper/2406.00611/) (2024)
- [Explainable Lifelong Stream Learning Based on "Glocal" Pairwise Fusion](https://mlsanity.com/paper/2306.13410/) (2023)
- [Learning from Exemplary Explanations](https://mlsanity.com/paper/2307.06026/) (2023)
- [Distance-Aware eXplanation Based Learning](https://mlsanity.com/paper/2309.05548/) (2023)
- [Are Logistic Models Really Interpretable?](https://mlsanity.com/paper/2406.13427/) (2024)
- [Rethinking Log Odds: Linear Probability Modelling and Expert Advice in Interpretable Machine Learning](https://mlsanity.com/paper/2211.06360/) (2022)
- [Testing the effectiveness of saliency-based explainability in NLP using randomized survey-based experiments](https://mlsanity.com/paper/2211.15351/) (2022)
- [Transitivity Recovering Decompositions: Interpretable and Robust Fine-Grained Relationships](https://mlsanity.com/paper/2310.15999/) (2023)
- [Explaining Neural Networks with Reasons](https://mlsanity.com/paper/2505.14424/) (2025)
- [Causal inference and model explainability tools for retail](https://mlsanity.com/paper/2512.12605/) (2025)
- [Interpretable rainfall modelling reveals rapid reorganisation of Amazonian rainfall under vegetation loss](https://mlsanity.com/paper/2605.10948/) (2026)
- [Integrating Feature Understandability into Feature Selection for Human-Centred Tabular Explanations](https://mlsanity.com/paper/2610.16627/) (2026)

## Abstract (arXiv)

To date, there has been no formal study of the statistical cost of interpretability in machine learning. As such, the discourse around potential trade-offs is often informal and misconceptions abound. In this work, we aim to initiate a formal study of these trade-offs. A seemingly insurmountable roadblock is the lack of any agreed upon definition of interpretability. Instead, we propose a shift in perspective. Rather than attempt to define interpretability, we propose to model the act of enforcing interpretability. As a starting point, we focus on the setting of empirical risk minimization for binary classification, and view interpretability as a constraint placed on learning. That is, we assume we are given a subset of hypothesis that are deemed to be interpretable, possibly depending on the data distribution and other aspects of the context. We then model the act of enforcing interpretability as that of performing empirical risk minimization over the set of interpretable hypotheses. This model allows us to reason about the statistical implications of enforcing interpretability, using known results in statistical learning theory. Focusing on accuracy, we perform a case analysis, explaining why one may or may not observe a trade-off between accuracy and interpretability when the restriction to interpretable classifiers does or does not come at the cost of some excess statistical risk. We close with some worked examples and some open problems, which we hope will spur further theoretical development around the tradeoffs involved in interpretability.