mlsanityData to 7 Oct 2026

How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation

arXiv 1802.00682 ↗First posted 2 Feb 2018cs.AIcited by 71

Not on the map. Why?

Abstract (arXiv)

Recent years have seen a boom in interest in machine learning systems that can provide a human-understandable rationale for their predictions or decisions. However, exactly what kinds of explanation are truly human-interpretable remains poorly understood. This work advances our understanding of what makes explanations interpretable in the specific context of verification. Suppose we have a machine learning system that predicts X, and we provide rationale for this prediction X. Given an input, an explanation, and an output, is the output consistent with the input and the supposed rationale? Via a series of user-studies, we identify what kinds of increases in complexity have the greatest effect on the time it takes for humans to verify the rationale, and which seem relatively insensitive.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 21; 2023: 14; 2024: 8; 2025: 5; 2026: 8.

71 papers.

Sort:

By area:

  1. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  2. 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 156
  3. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  4. 2022Explanations Can Reduce Overreliance on AI Systems During Decision-Makingcited by 111
  5. 2021Towards a Science of Human-AI Decision Making: A Survey of Empirical Studiescited by 85
  6. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  7. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 84
  8. 2021Towards Unbiased and Accurate Deferral to Multiple Expertscited by 71
  9. 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
  10. 2021How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanationscited by 58
  11. 2021Explanation-Based Human Debugging of NLP Models: A Surveycited by 53
  12. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
38 more
  1. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  2. 2022Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanationscited by 40
  3. 2022Missingness Bias in Model Debuggingcited by 36
  4. 2024Gradient based Feature Attribution in Explainable AI: A Technical Reviewcited by 34
  5. 2021Teaching Humans When To Defer to a Classifier via Exemplarscited by 29
  6. 2023Selective Explanations: Leveraging Human Input to Align Explainable AIcited by 29
  7. 2022Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Makingcited by 25
  8. 2024Properties and Challenges of LLM-Generated Explanationscited by 24
  9. 2022Computing Abductive Explanations for Boosted Treescited by 23
  10. 2022Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questionscited by 22
  11. 2023Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errorscited by 22
  12. 2022What Makes a Good Explanation?: A Harmonized View of Properties of Explanationscited by 21
  13. 2022The Manifold Hypothesis for Gradient-Based Explanationscited by 15
  14. 2021On the Interaction of Belief Bias and Explanationscited by 13
  15. 2023Explainability in AI Policies: A Critical Review of Communications, Reports, Regulations, and Standards in the EU, US, and UKcited by 13
  16. 2023On the Interpretability of Part-Prototype Based Classifiers: A Human Centric Analysiscited by 11
  17. 2022A Psychological Theory of Explainabilitycited by 10
  18. 2022Let's Go to the Alien Zoo: Introducing an Experimental Framework to Study Usability of Counterfactual Explanations for Machine Learningcited by 9
  19. 2024Incremental XAI: Memorable Understanding of AI with Incremental Explanationscited by 8
  20. 2021Crowdsourcing Evaluation of Saliency-based XAI Methodscited by 7
  21. 2025Improving Human Verification of LLM Reasoning through Interactive Explanation Interfacescited by 7
  22. 2021Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Studycited by 6
  23. 2022Towards Human-Compatible XAI: Explaining Data Differentials with Concept Induction over Background Knowledgecited by 6
  24. 2024What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blockscited by 6
  25. 2021Improving Users' Mental Model with Attention-directed Counterfactual Editscited by 5
  26. 2021On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI Systemcited by 5
  27. 2024How Human-Centered Explainable AI Interface Are Designed and Evaluated: A Systematic Surveycited by 5
  28. 2025VirtualXAI: A User-Centric Framework for Explainability Assessment Leveraging GPT-Generated Personascited by 5
  29. 2023Invisible Users: Uncovering End-Users' Requirements for Explainable AI via Explanation Forms and Goalscited by 4
  30. 2022Comparing Baseline Shapley and Integrated Gradients for Local Explanation: Some Additional Insightscited by 3
  31. 2023Helpful, Misleading or Confusing: How Humans Perceive Fundamental Building Blocks of Artificial Intelligence Explanationscited by 3
  32. 2021Explainable AI for medical imaging: Explaining pneumothorax diagnoses with Bayesian Teachingcited by 2
  33. 2022Transcending XAI Algorithm Boundaries through End-User-Inspired Designcited by 2
  34. 2023Good-looking but Lacking Faithfulness: Understanding Local Explanation Methods through Trend-based Testingcited by 2
  35. 2023Evaluation of Human-Understandability of Global Model Explanations using Decision Treecited by 2
  36. 2025See What I Mean? CUE: A Cognitive Model of Understanding Explanationscited by 2
  37. 2026Mixture of Concept Bottleneck Expertscited by 2
  38. 2022Can counterfactual explanations of AI systems' predictions skew lay users' causal intuitions about the world? If so, can we correct for that?cited by 1

50 most cited of 71 All 71 (JSON)