# Evaluating and Aggregating Feature-based Model Explanations

arXiv: https://arxiv.org/abs/2005.00631 · HTML: https://mlsanity.com/paper/2005.00631/

## Key facts

- Cited by 188 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2020.

## Built on by (most cited first, 30 of 188)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond](https://mlsanity.com/paper/2202.06861/) (2022)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- [A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation](https://mlsanity.com/paper/2306.07304/) (2023)
- [A Consistent and Efficient Evaluation Strategy for Attribution Methods](https://mlsanity.com/paper/2202.00449/) (2022)
- [What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods](https://mlsanity.com/paper/2112.04417/) (2021)
- [Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models](https://mlsanity.com/paper/2502.12892/) (2025)
- [Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis](https://mlsanity.com/paper/2111.04138/) (2021)
- [Synthetic Benchmarks for Scientific Research in Explainable Machine Learning](https://mlsanity.com/paper/2106.12543/) (2021)
- [Framework for Evaluating Faithfulness of Local Explanations](https://mlsanity.com/paper/2202.00734/) (2022)
- [Learning to Estimate Shapley Values with Vision Transformers](https://mlsanity.com/paper/2206.05282/) (2022)
- [Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations](https://mlsanity.com/paper/2205.07277/) (2022)
- [A Survey on the Robustness of Feature Importance and Counterfactual Explanations](https://mlsanity.com/paper/2111.00358/) (2021)
- [Don't Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysis](https://mlsanity.com/paper/2202.07728/) (2022)
- [The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus](https://mlsanity.com/paper/2302.07265/) (2023)
- [Explain Any Concept: Segment Anything Meets Concept-Based Explanation](https://mlsanity.com/paper/2305.10289/) (2023)
- [Making Sense of Dependence: Efficient Black-box Explanations Using Dependence Measure](https://mlsanity.com/paper/2206.06219/) (2022)
- [NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weights](https://mlsanity.com/paper/2106.10185/) (2021)
- [Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks](https://mlsanity.com/paper/2401.06654/) (2024)
- [Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science](https://mlsanity.com/paper/2303.00652/) (2023)
- [Xplique: A Deep Learning Explainability Toolbox](https://mlsanity.com/paper/2206.04394/) (2022)
- [What Makes a Good Explanation?: A Harmonized View of Properties of Explanations](https://mlsanity.com/paper/2211.05667/) (2022)
- [Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanations](https://mlsanity.com/paper/2211.12486/) (2022)
- [Fooling Partial Dependence via Data Poisoning](https://mlsanity.com/paper/2105.12837/) (2021)
- [A comprehensive study on fidelity metrics for XAI](https://mlsanity.com/paper/2401.10640/) (2024)
- [Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics](https://mlsanity.com/paper/2409.16756/) (2024)
- [Explainable Deep Learning in Healthcare: A Methodological Survey from an Attribution View](https://mlsanity.com/paper/2112.02625/) (2021)
- [Assessing Fidelity in XAI post-hoc techniques: A Comparative Study with Ground Truth Explanations Datasets](https://mlsanity.com/paper/2311.01961/) (2023)
- [Global Counterfactual Directions](https://mlsanity.com/paper/2404.12488/) (2024)
- [Understanding Visual Feature Reliance through the Lens of Complexity](https://mlsanity.com/paper/2407.06076/) (2024)

## Abstract (arXiv)

A feature-based model explanation denotes how much each input feature contributes to a model's output for a given data point. As the number of proposed explanation functions grows, we lack quantitative evaluation criteria to help practitioners know when to use which explanation function. This paper proposes quantitative evaluation criteria for feature-based explanations: low sensitivity, high faithfulness, and low complexity. We devise a framework for aggregating explanation functions. We develop a procedure for learning an aggregate explanation function with lower complexity and then derive a new aggregate Shapley value explanation function that minimizes sensitivity.