mlsanityData to 7 Oct 2026

The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus

arXiv 2302.07265 ↗First posted 14 Feb 2023cs.LG cs.AIcited by 37

Not on the map. Why?

Abstract (arXiv)

One of the unsolved challenges in the field of Explainable AI (XAI) is determining how to most reliably estimate the quality of an explanation method in the absence of ground truth explanation labels. Resolving this issue is of utmost importance as the evaluation outcomes generated by competing evaluation methods (or ''quality estimators''), which aim at measuring the same property of an explanation method, frequently present conflicting rankings. Such disagreements can be challenging for practitioners to interpret, thereby complicating their ability to select the best-performing explanation method. We address this problem through a meta-evaluation of different quality estimators in XAI, which we define as ''the process of evaluating the evaluation method''. Our novel framework, MetaQuantus, analyses two complementary performance characteristics of a quality estimator: its resilience to noise and reactivity to randomness, thus circumventing the need for ground truth labels. We demonstrate the effectiveness of our framework through a series of experiments, targeting various open questions in XAI such as the selection and hyperparameter optimisation of quality estimators. Our work is released under an open-source license (https://github.com/annahedstroem/MetaQuantus) to serve as a development tool for XAI- and Machine Learning (ML) practitioners to verify and benchmark newly constructed quality estimators in a given explainability context. With this work, we provide the community with clear and theoretically-grounded guidance for identifying reliable evaluation methods, thus facilitating reproducibility in the field.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 52 of its 69 references that are in the map. First posted on arXiv 14 Feb 2023.

Builds on

52 of 69 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  7. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  8. 2013Intriguing properties of neural networkscited by 5,114
  9. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  10. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  11. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  12. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
38 more
  1. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  2. 2018Sanity Checks for Saliency Mapscited by 1,149
  3. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  4. 2016The Mythos of Model Interpretabilitycited by 716
  5. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  6. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  7. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  8. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  9. 2016Top-down Neural Attention by Excitation Backpropcited by 318
  10. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  11. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  12. 2019Explanations can be manipulated and geometry is to blamecited by 215
  13. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  14. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  15. 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 156
  16. 2020Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAIcited by 125
  17. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
  18. 2019Sanity Checks for Saliency Metricscited by 119
  19. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
  20. 2019When Explanations Lie: Why Many Modified BP Attributions Failcited by 92
  21. 2019Benchmarking Attribution Methods with Relative Feature Importancecited by 89
  22. 2020On quantitative aspects of model interpretabilitycited by 71
  23. 2019Towards Best Practice in Explaining Neural Network Decisions with LRPcited by 70
  24. 2021Synthetic Benchmarks for Scientific Research in Explainable Machine Learningcited by 59
  25. 2022Framework for Evaluating Faithfulness of Local Explanationscited by 55
  26. 2022Rethinking Stability for Attribution-based Explanationscited by 52
  27. 2020IROF: a low resource evaluation metric for explanation methodscited by 45
  28. 2022Learning to Estimate Shapley Values with Vision Transformerscited by 44
  29. 2021Order in the Court: Explainable AI Methods Prone to Disagreementcited by 36
  30. 2021Interpretable Semantic Photo Geolocationcited by 32
  31. 2021NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weightscited by 28
  32. 2022Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanationscited by 22
  33. 2021Focus! Rating XAI Methods and Finding Biasescited by 20
  34. 2021Revisiting Sanity Checks for Saliency Mapscited by 20
  35. 2022Evaluating Feature Attribution Methods in the Image Domaincited by 19
  36. 2021Investigating sanity checks for saliency maps with image and text classificationcited by 11
  37. 2022Optimizing Explanations by Network Canonization and Hyperparameter Searchcited by 11
  38. 2018A Note about: Local Explanation Methods for Deep Neural Networks lack Sensitivity to Parameter Valuescited by 9

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 14; 2025: 5; 2026: 13.

37 papers.

Sort:

By area:

  1. 2023Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Sciencecited by 25
  2. 2024CoSy: Evaluating Textual Explanations of Neuronscited by 17
  3. 2024A comprehensive study on fidelity metrics for XAIcited by 14
  4. 2023Class-Discriminative Attention Maps for Vision Transformerscited by 10
  5. 2024Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Testcited by 9
  6. 2024A Fresh Look at Sanity Checks for Saliency Mapscited by 8
  7. 2024FLEXtime: Filterbank learning to explain time seriescited by 6
  8. 2024T-TAME: Trainable Attention Mechanism for Explaining Convolutional Networks and Vision Transformerscited by 4
  9. 2024Expected Grad-CAM: Towards gradient faithfulnesscited by 4
  10. 2024FreqRISE: Explaining time series using frequency maskingcited by 4
  11. 2024Explainable AI needs formalizationcited by 4
  12. 2024From Flexibility to Manipulation: The Slippery Slope of XAI Evaluationcited by 4
25 more
  1. 2025On the Effectiveness of Methods and Metrics for Explainable AI in Remote Sensing Image Scene Classificationcited by 3
  2. 2023How Well Do Feature-Additive Explainers Explain Feature-Additive Predictors?cited by 2
  3. 2026Feature salience - not task-informativeness - drives machine learning model explanationscited by 2
  4. 2026Evaluating the Interpretability of Sparse Autoencoders with Concept Annotationscited by 2
  5. 2023Is Task-Agnostic Explainable AI a Myth?cited by 1
  6. 2024CNN-based explanation ensembling for dataset, representation and explanations evaluationcited by 1
  7. 2024REPEAT: Improving Uncertainty Estimation in Representation Learning Explainabilitycited by 1
  8. 2025Back to the Baseline: Examining Baseline Effects on Explainability Metricscited by 1
  9. 2026LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactualscited by 1
  10. 2026Exploring SAIG Methods for an Objective Evaluation of XAIcited by 1
  11. 2023Explainable AI in Grassland Monitoring: Enhancing Model Performance and Domain Adaptabilitycited by 0
  12. 2024NormEnsembleXAI: Unveiling the Strengths and Weaknesses of XAI Ensemble Techniquescited by 0
  13. 2024Why the Agent Made that Decision: Contrastive Explanation Learning for Reinforcement Learningcited by 0
  14. 2025Evaluate with the Inverse: Efficient Approximation of Latent Explanation Quality Distributioncited by 0
  15. 2025Rethinking Explanation Evaluation under the Retraining Schemecited by 0
  16. 2025φ-Table: A Statistical Explanation for Global SHAPcited by 0
  17. 2026Does Explanation Correctness Matter? Linking Computational XAI Evaluation to Human Understandingcited by 0
  18. 2026Structural Compactness as a Complementary Criterion for Explanation Qualitycited by 0
  19. 2026Quantifying Explanation Consistency: The C-Score Metric for CAM-Based Explainability in Medical Image Classificationcited by 0
  20. 2026When Fine-Tuning Changes the Evidence: Architecture-Dependent Semantic Drift in Chest X-Ray Explanationscited by 0
  21. 2026Quantifying Explainable AI-introduced signal noise on ECG data with Spectral Entropycited by 0
  22. 2026Position: Explanation Stability Is a Property of the Model Method Pair, Not the Modelcited by 0
  23. 2026Explaining AI-Image Detection: What the Heatmap Actually Showscited by 0
  24. 2026Learnable Time-Frequency Masks for Explaining Time-Series Classifierscited by 0
  25. 2026A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methodscited by 0