# Measuring Explainer Stability via Attribution Separability

arXiv: https://arxiv.org/abs/2608.02697 · HTML: https://mlsanity.com/paper/2608.02697/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Builds on (most cited first, 22 of 22)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [Techniques for Interpretable Machine Learning](https://mlsanity.com/paper/1808.00033/) (2018)
- [Explanations can be manipulated and geometry is to blame](https://mlsanity.com/paper/1906.07983/) (2019)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regression](https://mlsanity.com/paper/2012.01536/) (2020)
- [Benchmarking and Survey of Explanation Methods for Black Box Models](https://mlsanity.com/paper/2102.13076/) (2021)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [Rethinking Stability for Attribution-based Explanations](https://mlsanity.com/paper/2203.06877/) (2022)
- [Gradient based Feature Attribution in Explainable AI: A Technical Review](https://mlsanity.com/paper/2403.10415/) (2024)
- [Logic Traps in Evaluating Attribution Scores](https://mlsanity.com/paper/2109.05463/) (2021)
- [When are Post-hoc Conceptual Explanations Identifiable?](https://mlsanity.com/paper/2206.13872/) (2022)
- [Stability Guarantees for Feature Attributions with Multiplicative Smoothing](https://mlsanity.com/paper/2307.05902/) (2023)
- [Stabilizing Estimates of Shapley Values with Control Variates](https://mlsanity.com/paper/2310.07672/) (2023)
- [Statistical Significance of Feature Importance Rankings](https://mlsanity.com/paper/2401.15800/) (2024)
- [An Empirical Comparison of Explainable Artificial Intelligence Methods for Clinical Data: A Case Study on Traumatic Brain Injury](https://mlsanity.com/paper/2208.06717/) (2022)
- [CID: Measuring Feature Importance Through Counterfactual Distributions](https://mlsanity.com/paper/2511.15371/) (2025)

## Abstract (arXiv)

Attribution methods (AMs) assign an importance score to each feature and are widely adopted to explain black-box models. However, most methods can produce variable attribution scores due to stochastic components in their definition. In this paper, we propose a distribution-based framework to capture the stability of attribution scores. In particular, our approach allows to understand the degree of separability in the ranked attribution vector and obtain the largest index for which a feature ranking remains reliable. We further extend this framework to compare AMs based on the robustness of their rankings across a dataset. Through experiments, we demonstrate how to apply our method to evaluate explainer stability. Overall, our approach provides a complementary criterion for evaluating the stability of AMs.