mlsanityData to 14 Oct 2026

Evaluating Model Explanations without Ground Truth

arXiv 2505.10399 ↗First posted 15 May 2025cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

There can be many competing and contradictory explanations for a single model prediction, making it difficult to select which one to use. Current explanation evaluation frameworks measure quality by comparing against ideal "ground-truth" explanations, or by verifying model sensitivity to important inputs. We outline the limitations of these approaches, and propose three desirable principles to ground the future development of explanation evaluation strategies for local feature importance explanations. We propose a ground-truth Agnostic eXplanation Evaluation framework (AXE) for evaluating and comparing model explanations that satisfies these principles. Unlike prior approaches, AXE does not require access to ideal ground-truth explanations for comparison, or rely on model sensitivity - providing an independent measure of explanation quality. We verify AXE by comparing with baselines, and show how it can be used to detect explanation fairwashing. Our code is available at https://github.com/KaiRawal/Evaluating-Model-Explanations-without-Ground-Truth.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 33 of its 58 references that are in the map. First posted on arXiv 15 May 2025.

Builds on

33 of 58 references.

References · 33

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  6. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  7. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  8. 2018Sanity Checks for Saliency Mapscited by 1,156
  9. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  10. 2016The Mythos of Model Interpretabilitycited by 721
  11. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  12. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
21 more
  1. 2016Top-down Neural Attention by Excitation Backpropcited by 321
  2. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  3. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
  4. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  5. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 190
  6. 2019Predictive Multiplicity in Classificationcited by 166
  7. 2019Learning Model-Agnostic Counterfactual Explanations for Tabular Datacited by 146
  8. 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 139
  9. 2021Human-Centered Explainable AI (XAI): From Algorithms to User Experiencescited by 123
  10. 2020Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challengescited by 120
  11. 2019XRAI: Better Attributions Through Regionscited by 116
  12. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  13. 2020Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Studycited by 86
  14. 2021Synthetic Benchmarks for Scientific Research in Explainable Machine Learningcited by 59
  15. 2018Do Explanations make VQA Models more Predictable to a Human?cited by 52
  16. 2022Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanationscited by 40
  17. 2022Machine Explanations and Human Understandingcited by 20
  18. 2022Stability of clinical prediction models developed using statistical or machine learning methodscited by 12
  19. 2024Classification Metrics for Image Explanations: Towards Building Reliable XAI-Evaluationscited by 7
  20. 2023The Disagreement Problem in Faithfulness Metricscited by 6
  21. 2024Understanding Disparities in Post Hoc Machine Learning Explanationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2025Variance-Bounded Evaluation of Entity-Centric AI Systems Without Ground Truth: Theory and Measurementcited by 2
  2. 2026xaitimesynth: A Python Package for Evaluating Attribution Methods for Time Series with Synthetic Ground Truthcited by 1
  3. 2026Implicit Hypothesis Testing and Divergence Preservation in Neural Network Representationscited by 0
  4. 2026Does Explanation Correctness Matter? Linking Computational XAI Evaluation to Human Understandingcited by 0
  5. 2026Stabilising Explainability Fragility in Cybersecurity AI: The Impact and Mitigation of Multicollinearity in Public Benchmark Datasetscited by 0
  6. 2026FMMO: Detecting the Divergence Between Local Attribution and Global Driftcited by 0
  7. 2026MEA: A Reward-Driven Multi-Agent System for Faithful Model Explanationscited by 0