mlsanityData to 7 Oct 2026

Framework for Evaluating Faithfulness of Local Explanations

arXiv 2202.00734 ↗First posted 1 Feb 2022cs.LG stat.MLcited by 55

Not on the map. Why?

Abstract (arXiv)

We study the faithfulness of an explanation system to the underlying prediction model. We show that this can be captured by two properties, consistency and sufficiency, and introduce quantitative measures of the extent to which these hold. Interestingly, these measures depend on the test-time data distribution. For a variety of existing explanation systems, such as anchors, we analytically study these quantities. We also provide estimators and sample complexity bounds for empirically determining the faithfulness of black-box explanation systems. Finally, we experimentally validate the new properties and estimators.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 38 of its 59 references that are in the map. First posted on arXiv 1 Feb 2022.

Builds on

38 of 59 references.

References · 38

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  5. 2018Sanity Checks for Saliency Mapscited by 1,149
  6. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  7. 2016The Mythos of Model Interpretabilitycited by 716
  8. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 612
  9. 2018On the Robustness of Interpretability Methodscited by 310
  10. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  11. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  12. 2018Manipulating and Measuring Model Interpretabilitycited by 256
26 more
  1. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  2. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  3. 2019Sanity Checks for Saliency Metricscited by 119
  4. 2021Counterfactual Explanations Can Be Manipulatedcited by 113
  5. 2019Optimal Sparse Decision Treescited by 104
  6. 2020Evaluating Explanations: How much do explanations from the teacher aid students?cited by 94
  7. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 86
  8. 2020Aligning Faithful Interpretations with their Social Attributioncited by 79
  9. 2020Model Interpretability through the Lens of Computational Complexitycited by 75
  10. 2020Fairwashing Explanations with Off-Manifold Detergentcited by 71
  11. 2020Explainable k-Means and k-Medians Clusteringcited by 67
  12. 2018A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizationscited by 59
  13. 2021How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanationscited by 58
  14. 2020Towards falsifiable interpretability researchcited by 53
  15. 2016Learning Optimized Risk Scorescited by 47
  16. 2020Robust and Stable Black Box Explanationscited by 47
  17. 2021Towards the Unification and Robustness of Perturbation and Gradient Based Explanationscited by 47
  18. 2020Gradient-based Analysis of NLP Models is Manipulablecited by 41
  19. 2020How Good is your Explanation? Algorithmic Stability Measures to Assess the Quality of Explanations for Deep Neural Networkscited by 27
  20. 2021To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methodscited by 26
  21. 2021Revisiting The Evaluation of Class Activation Mapping for Explainability: A Novel Metric and Experimental Analysiscited by 25
  22. 2018A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learningcited by 21
  23. 2021Connecting Interpretability and Robustness in Decision Trees through Separationcited by 16
  24. 2020A Formal Approach to Explainabilitycited by 9
  25. 2021Developing a Fidelity Evaluation Approach for Interpretable Machine Learningcited by 8
  26. 2020A New Method to Compare the Interpretability of Rule-based Algorithmscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 9; 2024: 15; 2025: 18; 2026: 8.

55 papers.

Sort:

By area:

  1. 2023Explainability for Large Language Models: A Surveycited by 331
  2. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  3. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
  4. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  5. 2023Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistencycited by 42
  6. 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
  7. 2022MultiViz: Towards Visualizing and Understanding Multimodal Modelscited by 36
  8. 2022SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretabilitycited by 28
  9. 2024Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metricscited by 15
  10. 2023Feature Perturbation Augmentation for Reliable Evaluation of Importance Estimators in Neural Networkscited by 9
  11. 2023A Framework for Interpretability in Machine Learning for Medical Imagingcited by 9
  12. 2024Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Testcited by 9
38 more
  1. 2024Intrinsic User-Centric Interpretability through Global Mixture of Expertscited by 9
  2. 2024A Fresh Look at Sanity Checks for Saliency Mapscited by 8
  3. 2024Auditing Local Explanations is Hardcited by 8
  4. 2024Benchmarking XAI Explanations with Human-Aligned Evaluationscited by 8
  5. 2023Proto-lm: A Prototypical Network-Based Framework for Built-in Interpretability in Large Language Modelscited by 7
  6. 2022XAudit : A Theoretical Look at Auditing with Explanationscited by 5
  7. 2024Toward Understanding the Disagreement Problem in Neural Network Feature Attributioncited by 5
  8. 2023A Survey of Explainable AI in Deep Visual Modeling: Methods and Metricscited by 4
  9. 2024Expected Grad-CAM: Towards gradient faithfulnesscited by 4
  10. 2024From Flexibility to Manipulation: The Slippery Slope of XAI Evaluationcited by 4
  11. 2025Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency mapscited by 4
  12. 2024IBO: Inpainting-Based Occlusion to Enhance Explainable Artificial Intelligence Evaluation in Histopathologycited by 3
  13. 2026GNN Explanations that do not Explain and How to find Themcited by 3
  14. 2023Faithful and Robust Local Interpretability for Textual Predictionscited by 2
  15. 2024A Sim2Real Approach for Identifying Task-Relevant Properties in Interpretable Machine Learningcited by 2
  16. 2024DISCRET: Synthesizing Faithful Explanations For Treatment Effect Estimationcited by 2
  17. 2024GIFT: A Framework Towards Global Interpretable Faithful Textual Explanations of Vision Classifierscited by 2
  18. 2025Privacy Risks and Preservation Methods in Explainable Artificial Intelligence: A Scoping Reviewcited by 2
  19. 2025Argument-Based Consistency in Toxicity Explanations of LLMscited by 2
  20. 2025Knowing What You Cannot Explain: Learning to Reject Low-Quality Explanationscited by 2
  21. 2025Informative Post-Hoc Explanations Only Exist for Simple Functionscited by 2
  22. 2026Position: Explainability Research Must Prioritize Foundations over Ad-hoc Methodscited by 2
  23. 2024Advancing Attribution-Based Neural Network Explainability through Relative Absolute Magnitude Layer-Wise Relevance Propagation and Multi-Component Evaluationcited by 1
  24. 2024Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquirycited by 1
  25. 2025ExpProof : Operationalizing Explanations for Confidential Models with ZKPscited by 1
  26. 2025How to safely discard features based on aggregate SHAP valuescited by 1
  27. 2025Towards an Evaluation Framework for Explainable Artificial Intelligence Systems for Health and Well-beingcited by 1
  28. 2025Beyond Patches: Mining Interpretable Part-Prototypes for Explainable AIcited by 1
  29. 2025Explanations Go Linear: Post-hoc Explainability for Tabular Data with Interpretable Meta-Encodingcited by 1
  30. 2025Towards integration of Privacy Enhancing Technologies in Explainable Artificial Intelligencecited by 1
  31. 2026Unifying Formal Explanations: A Complexity-Theoretic Perspectivecited by 1
  32. 2025When Machine Learning Gets Personal: Evaluating Prediction and Explanationcited by 0
  33. 2025Evaluate with the Inverse: Efficient Approximation of Latent Explanation Quality Distributioncited by 0
  34. 2025DeepFaith: A Domain-Free and Model-Agnostic Unified Framework for Highly Faithful Explanationscited by 0
  35. 2025Assessing the Noise Robustness of Class Activation Maps: A Framework for Reliable Model Interpretabilitycited by 0
  36. 2025Value bounds and Convergence Analysis for Averages of LRP attributionscited by 0
  37. 2025Additive Large Language Models for Semi-Structured Textcited by 0
  38. 2025φ-Table: A Statistical Explanation for Global SHAPcited by 0

50 most cited of 55 All 55 (JSON)