# Learning outside the Black-Box: The pursuit of interpretable models

arXiv: https://arxiv.org/abs/2011.08596 · HTML: https://mlsanity.com/paper/2011.08596/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2020.

## Built on by (most cited first, 18 of 18)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [On the Computational Intelligibility of Boolean Classifiers](https://mlsanity.com/paper/2104.06172/) (2021)
- [Precise characterization of the prior predictive distribution of deep ReLU networks](https://mlsanity.com/paper/2106.06615/) (2021)
- [Deep Generative Symbolic Regression](https://mlsanity.com/paper/2401.00282/) (2023)
- [Label-Free Explainability for Unsupervised Models](https://mlsanity.com/paper/2203.01928/) (2022)
- [On (assessing) the fairness of risk score models](https://mlsanity.com/paper/2302.08851/) (2023)
- [ODE Discovery for Longitudinal Heterogeneous Treatment Effects Inference](https://mlsanity.com/paper/2403.10766/) (2024)
- [AutoPrognosis 2.0: Democratizing Diagnostic and Prognostic Modeling in Healthcare with Automated Machine Learning](https://mlsanity.com/paper/2210.12090/) (2022)
- [Symbolic Recovery of Differential Equations: The Identifiability Problem](https://mlsanity.com/paper/2210.08342/) (2022)
- [D-CIPHER: Discovery of Closed-form Partial Differential Equations](https://mlsanity.com/paper/2206.10586/) (2022)
- [Closed-Form Interpretation of Neural Network Classifiers with Symbolic Gradients](https://mlsanity.com/paper/2401.04978/) (2024)
- [Symbolic Metamodels for Interpreting Black-boxes Using Primitive Functions](https://mlsanity.com/paper/2302.04791/) (2023)
- [How Well Do Feature-Additive Explainers Explain Feature-Additive Predictors?](https://mlsanity.com/paper/2310.18496/) (2023)
- [AdaptGrad: Adaptive Sampling to Reduce Noise](https://mlsanity.com/paper/2410.07711/) (2024)
- [Interpreting deep embeddings for disease progression clustering](https://mlsanity.com/paper/2307.06060/) (2023)
- [Axiomatization of Gradient Smoothing in Neural Networks](https://mlsanity.com/paper/2407.00371/) (2024)
- [ITL-LIME: Instance-Based Transfer Learning for Enhancing Local Explanations in Low-Resource Data Settings](https://mlsanity.com/paper/2508.13672/) (2025)
- [MorphingNet: Symbolic Regression Where the Network Is the Formula](https://mlsanity.com/paper/2610.19942/) (2026)

## Abstract (arXiv)

Machine Learning has proved its ability to produce accurate models but the deployment of these models outside the machine learning community has been hindered by the difficulties of interpreting these models. This paper proposes an algorithm that produces a continuous global interpretation of any given continuous black-box function. Our algorithm employs a variation of projection pursuit in which the ridge functions are chosen to be Meijer G-functions, rather than the usual polynomial splines. Because Meijer G-functions are differentiable in their parameters, we can tune the parameters of the representation by gradient descent; as a consequence, our algorithm is efficient. Using five familiar data sets from the UCI repository and two familiar machine learning algorithms, we demonstrate that our algorithm produces global interpretations that are both highly accurate and parsimonious (involve a small number of terms). Our interpretations permit easy understanding of the relative importance of features and feature interactions. Our interpretation algorithm represents a leap forward from the previous state of the art.