mlsanityData to 7 Oct 2026

Benchmarking XAI Explanations with Human-Aligned Evaluations

arXiv 2411.02470 ↗First posted 4 Nov 2024cs.CV cs.AI cs.HCcited by 8

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

We introduce PASTA (Perceptual Assessment System for explanaTion of Artificial Intelligence), a novel human-centric framework for evaluating eXplainable AI (XAI) techniques in computer vision. Our first contribution is the creation of the PASTA-dataset, the first large-scale benchmark that spans a diverse set of models and both saliency-based and concept-based explanation methods. This dataset enables robust, comparative analysis of XAI techniques based on human judgment. Our second contribution is an automated, data-driven benchmark that predicts human preferences using the PASTA-dataset. This scoring called PASTA-score method offers scalable, reliable, and consistent evaluation aligned with human perception. Additionally, our benchmark allows for comparisons between explanations across different modalities, an aspect previously unaddressed. We then propose to apply our scoring method to probe the interpretability of existing models and to build more human interpretable XAI methods.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 76 of its 119 references that are in the map. First posted on arXiv 4 Nov 2024.

Builds on

76 of 119 references.

References · 76

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  8. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  9. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  10. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  11. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  12. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
38 more
  1. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  2. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  3. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  4. 2020Concept Bottleneck Modelscited by 1,022
  5. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  6. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
  7. 2023Vision Transformers Need Registerscited by 844
  8. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  9. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  10. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  11. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  12. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  13. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  14. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  15. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  16. 2018On the Robustness of Interpretability Methodscited by 310
  17. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 310
  18. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  19. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  20. 2022Post-hoc Concept Bottleneck Modelscited by 289
  21. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  22. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 230
  23. 2019Full-Gradient Representation for Neural Network Visualizationcited by 189
  24. 2020Eigen-CAM: Class Activation Map using Principal Componentscited by 188
  25. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  26. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  27. 2019One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniquescited by 164
  28. 2020Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNscited by 151
  29. 2020Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAIcited by 125
  30. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
  31. 2024Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluationcited by 123
  32. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
  33. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  34. 2022Diffusion Visual Counterfactual Explanationscited by 96
  35. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 86
  36. 2020Use HiResCAM instead of Grad-CAM for faithful explanations of convolutional neural networkscited by 79
  37. 2023Learning Concise and Descriptive Attributes for Visual Recognitioncited by 73
  38. 2020On quantitative aspects of model interpretabilitycited by 71

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025Evaluating Neuron Explanations: A Unified Framework with Sanity Checkscited by 4
  2. 2025Knowing What You Cannot Explain: Learning to Reject Low-Quality Explanationscited by 2
  3. 2026Disentangling Hallucinations: Orthogonal Semantic Projection for Robust Interpretabilitycited by 1
  4. 2025Tuning for Trustworthiness -- Balancing Performance and Explanation Consistency in Neural Network Optimizationcited by 0
  5. 2026Structural Compactness as a Complementary Criterion for Explanation Qualitycited by 0
  6. 2026Improving Adversarial Robustness of Attribution via Implicit Regularizationcited by 0
  7. 2026Position: Explanation Stability Is a Property of the Model Method Pair, Not the Modelcited by 0
  8. 2026CHOQOLATE: Organizing Concept Bottleneck Latent Spaces with Choquet Integralscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Beyond the Veil of Similarity: Quantifying Semantic Continuity in Explainable AI
  2. 2023Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directions
  3. 2026Adapting Vision-Language Models for Human-Readable XAI in Industrial Object Detection
  4. 2026QQJ: Quantifying Qualitative Judgment for Scalable and Human-Aligned Evaluation of Generative AI
  5. 2025Your Reasoning Benchmark May Not Test Reasoning: Revealing Perception Bottleneck in Abstract Reasoning Benchmarks
  6. 2025A Survey on Interpretability in Visual Recognition
  7. 2025Explainable artificial intelligence (XAI): from inherent explainability to large language models
  8. 2022Towards Human-Interpretable Prototypes for Visual Assessment of Image Classification Models