mlsanityData to 14 Oct 2026

The Promise and Peril of Human Evaluation for Model Interpretability

arXiv 1711.07414 ↗First posted 20 Nov 2017cs.AI cs.LG stat.MLcited by 52

Not on the map. Why?

Abstract (arXiv)

Transparency, user trust, and human comprehension are popular ethical motivations for interpretable machine learning. In support of these goals, researchers evaluate model explanation performance using humans and real world applications. This alone presents a challenge in many areas of artificial intelligence. In this position paper, we propose a distinction between descriptive and persuasive explanations. We discuss reasoning suggesting that functional interpretability may be correlated with cognitive function and user preferences. If this is indeed the case, evaluation and optimization using functional metrics could perpetuate implicit cognitive bias in explanations that threaten transparency. Finally, we propose two potential research directions to disambiguate cognitive function and explanation models, retaining control over the tradeoff between accuracy and interpretability.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 9; 2023: 13; 2024: 16; 2025: 3; 2026: 2.

52 papers.

Sort:

By area:

  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  2. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  3. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  4. 2023Faithful Chain-of-Thought Reasoningcited by 288
  5. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  6. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  7. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  8. 2023Quantifying Context Mixing in Transformerscited by 43
  9. 2023ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writingcited by 31
  10. 2024Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuningcited by 26
  11. 2024Mechanistic?cited by 25
  12. 2024Properties and Challenges of LLM-Generated Explanationscited by 24
38 more
  1. 2023A Critical Survey on Fairness Benefits of Explainable AIcited by 16
  2. 2021Counterfactual Evaluation for Explainable AIcited by 15
  3. 2022A Fine-grained Interpretability Evaluation Benchmark for Neural NLPcited by 15
  4. 2021Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligencecited by 13
  5. 2023Improving Interpretation Faithfulness for Vision Transformerscited by 12
  6. 2023Towards Visual Saliency Explanations of Face Verificationcited by 11
  7. 2023Why is plausibility surprisingly problematic as an XAI criterion?cited by 9
  8. 2021Quality Metrics for Transparent Machine Learning With and Without Humans In the Loop Are Not Correlatedcited by 7
  9. 2022Mediators: Conversational Agents Explaining NLP Model Behaviorcited by 7
  10. 2024FLARE: Faithful Logic-Aided Reasoning and Explorationcited by 7
  11. 2022An Attention Matrix for Every Decision: Faithfulness-based Arbitration Among Multiple Attention-Based Interpretations of Transformers in Text Classificationcited by 6
  12. 2024What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blockscited by 6
  13. 2021Zero-shot Sequence Labeling for Transformer-based Sentence Classifierscited by 5
  14. 2021DuTrust: A Sentiment Analysis Dataset for Trustworthiness Evaluationcited by 5
  15. 2026Do explanations generalize across large reasoning models?cited by 5
  16. 2022Explainable Artificial Intelligence: Precepts, Methods, and Opportunities for Research in Constructioncited by 4
  17. 2024Do Robot Snakes Dream like Electric Sheep? Investigating the Effects of Architectural Inductive Biases on Hallucinationcited by 4
  18. 2021A Framework for Evaluating Post Hoc Feature-Additive Explainerscited by 3
  19. 2022Interpretation Quality Score for Measuring the Quality of interpretability methodscited by 3
  20. 2022A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challengescited by 3
  21. 2023Towards Faithful Knowledge Graph Explanation Through Deep Alignment in Commonsense Question Answeringcited by 3
  22. 2024Explainable Molecular Property Prediction: Aligning Chemical Concepts with Predictions via Language Modelscited by 3
  23. 2024Explanation sensitivity to the randomness of large language models: the case of journalistic text classificationcited by 3
  24. 2024Faithful Interpretation for Graph Neural Networkscited by 3
  25. 2021Faithful and Plausible Explanations of Medical Code Predictionscited by 2
  26. 2023Benchmarking Faithfulness: Towards Accurate Natural Language Explanations in Vision-Language Taskscited by 2
  27. 2023Looking deeper into interpretable deep learning in neuroimaging: a comprehensive surveycited by 2
  28. 2024XRL-Bench: A Benchmark for Evaluating and Comparing Explainable Reinforcement Learning Techniquescited by 2
  29. 2024Understanding the Dataset Practitioners Behind Large Language Model Developmentcited by 2
  30. 2024CoProNN: Concept-based Prototypical Nearest Neighbors for Explaining Vision Modelscited by 2
  31. 2021TDM: Trustworthy Decision-Making via Interpretability Enhancementcited by 1
  32. 2024Mapping the Potential of Explainable AI for Fairness Along the AI Lifecyclecited by 1
  33. 2024Towards A Comprehensive Visual Saliency Explanation Framework for AI-based Face Recognition Systemscited by 1
  34. 2025Thinking About Thinking: Evaluating Reasoning in Post-Trained Language Modelscited by 1
  35. 2023Survey of Trustworthy AI: A Meta Decision of AIcited by 0
  36. 2023Interpretability is not Explainability: New Quantitative XAI Approach with a focus on Recommender Systems in Educationcited by 0
  37. 2024Confident Teacher, Confident Student? A Novel User Study Design for Investigating the Didactic Potential of Explanations and their Impact on Uncertaintycited by 0
  38. 2024A Multi-Task Text Classification Pipeline with Natural Language Explanations: A User-Centric Evaluation in Sentiment Analysis and Offensive Language Identification in Greek Tweetscited by 0

50 most cited of 52 All 52 (JSON)