# Evaluating Explainability in Machine Learning Predictions through Explainer-Agnostic Metrics

arXiv: https://arxiv.org/abs/2302.12094 · HTML: https://mlsanity.com/paper/2302.12094/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 23 Feb 2023.

## Builds on (most cited first, 10 of 10)

- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Metrics for Explainable AI: Challenges and Prospects](https://mlsanity.com/paper/1812.04608/) (2018)
- [Benchmarking and Survey of Explanation Methods for Black Box Models](https://mlsanity.com/paper/2102.13076/) (2021)
- [How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation](https://mlsanity.com/paper/1802.00682/) (2018)
- [Towards Unifying Feature Attribution and Counterfactual Explanations: Different Means to the Same End](https://mlsanity.com/paper/2011.04917/) (2020)
- [Interpretable & Explorable Approximations of Black Box Models](https://mlsanity.com/paper/1707.01154/) (2017)
- [Quantifying Interpretability and Trust in Machine Learning Systems](https://mlsanity.com/paper/1901.08558/) (2019)
- [How Good is your Explanation? Algorithmic Stability Measures to Assess the Quality of Explanations for Deep Neural Networks](https://mlsanity.com/paper/2009.04521/) (2020)
- [Leveraging Latent Features for Local Explanations](https://mlsanity.com/paper/1905.12698/) (2019)

## Built on by (most cited first, 1 of 1)

- [Decoding Guardrails: XAI-Guided Perturbation Analysis of Prompt Injection Detection](https://mlsanity.com/paper/2609.24801/) (2026)

## Abstract (arXiv)

The rapid integration of artificial intelligence (AI) into various industries has introduced new challenges in governance and regulation, particularly regarding the understanding of complex AI systems. A critical demand from decision-makers is the ability to explain the results of machine learning models, which is essential for fostering trust and ensuring ethical AI practices. In this paper, we develop six distinct model-agnostic metrics designed to quantify the extent to which model predictions can be explained. These metrics measure different aspects of model explainability, ranging from local importance, global importance, and surrogate predictions, allowing for a comprehensive evaluation of how models generate their outputs. Furthermore, by computing our metrics, we can rank models in terms of explainability criteria such as importance concentration and consistency, prediction fluctuation, and surrogate fidelity and stability, offering a valuable tool for selecting models based not only on accuracy but also on transparency. We demonstrate the practical utility of these metrics on classification and regression tasks, and integrate these metrics into an existing Python package for public use.