# How Good is your Explanation? Algorithmic Stability Measures to Assess the Quality of Explanations for Deep Neural Networks

arXiv: https://arxiv.org/abs/2009.04521 · HTML: https://mlsanity.com/paper/2009.04521/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Sep 2020.

## Built on by (most cited first, 27 of 27)

- [A Consistent and Efficient Evaluation Strategy for Attribution Methods](https://mlsanity.com/paper/2202.00449/) (2022)
- [FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcare](https://mlsanity.com/paper/2309.12325/) (2023)
- [Harmonizing the object recognition strategies of deep neural networks with humans](https://mlsanity.com/paper/2211.04533/) (2022)
- [Framework for Evaluating Faithfulness of Local Explanations](https://mlsanity.com/paper/2202.00734/) (2022)
- [Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagation](https://mlsanity.com/paper/2210.07394/) (2022)
- [Revisiting Sanity Checks for Saliency Maps](https://mlsanity.com/paper/2110.14297/) (2021)
- [Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Set](https://mlsanity.com/paper/2110.13369/) (2021)
- [Benchmarking XAI Explanations with Human-Aligned Evaluations](https://mlsanity.com/paper/2411.02470/) (2024)
- [On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective](https://mlsanity.com/paper/2206.06854/) (2022)
- [Analyzing Explainer Robustness via Probabilistic Lipschitzness of Prediction Functions](https://mlsanity.com/paper/2206.12481/) (2022)
- [On the Robustness of Explanations of Deep Neural Network Models: A Survey](https://mlsanity.com/paper/2211.04780/) (2022)
- [Revisiting the robustness of post-hoc interpretability methods](https://mlsanity.com/paper/2407.19683/) (2024)
- [Don't Get Me Wrong: How to Apply Deep Visual Interpretations to Time Series](https://mlsanity.com/paper/2203.07861/) (2022)
- [On Minimizing the Impact of Dataset Shifts on Actionable Explanations](https://mlsanity.com/paper/2306.06716/) (2023)
- [Saliency strikes back: How filtering out high frequencies improves white-box explanations](https://mlsanity.com/paper/2307.09591/) (2023)
- [COSE: A Consistency-Sensitivity Metric for Saliency on Image Classification](https://mlsanity.com/paper/2309.10989/) (2023)
- [Axiomatic Explainer Globalness via Optimal Transport](https://mlsanity.com/paper/2411.01126/) (2024)
- [A Survey on Interpretability in Visual Recognition](https://mlsanity.com/paper/2507.11099/) (2025)
- [Evaluating Explainability in Machine Learning Predictions through Explainer-Agnostic Metrics](https://mlsanity.com/paper/2302.12094/) (2023)
- [Unlearning-based Neural Interpretations](https://mlsanity.com/paper/2410.08069/) (2024)
- [Gaussian Smoothing in Saliency Maps: The Stability-Fidelity Trade-Off in Neural Network Interpretability](https://mlsanity.com/paper/2411.05837/) (2024)
- [Metric Tools for Sensitivity Analysis with Applications to Neural Networks](https://mlsanity.com/paper/2305.02368/) (2023)
- [Verifying Global Neural Network Specifications using Hyperproperties](https://mlsanity.com/paper/2306.12495/) (2023)
- [Rethinking Robustness: A New Approach to Evaluating Feature Attribution Methods](https://mlsanity.com/paper/2512.06665/) (2025)
- [A Learning Stability Profile for Finite-Dimensional Learning Dynamics](https://mlsanity.com/paper/2512.21208/) (2025)
- [Toward Faithful Segmentation Attribution via Benchmarking and Dual-Evidence Fusion](https://mlsanity.com/paper/2603.22624/) (2026)
- [Empirical Characterization of Rationale Stability Under Controlled Perturbations for Explainable Pattern Recognition](https://mlsanity.com/paper/2604.04456/) (2026)

## Abstract (arXiv)

A plethora of methods have been proposed to explain how deep neural networks reach their decisions but comparatively, little effort has been made to ensure that the explanations produced by these methods are objectively relevant. While several desirable properties for trustworthy explanations have been formulated, objective measures have been harder to derive. Here, we propose two new measures to evaluate explanations borrowed from the field of algorithmic stability: mean generalizability MeGe and relative consistency ReCo. We conduct extensive experiments on different network architectures, common explainability methods, and several image datasets to demonstrate the benefits of the proposed measures.In comparison to ours, popular fidelity measures are not sufficient to guarantee trustworthy explanations.Finally, we found that 1-Lipschitz networks produce explanations with higher MeGe and ReCo than common neural networks while reaching similar accuracy. This suggests that 1-Lipschitz networks are a relevant direction towards predictors that are more explainable and trustworthy.