Towards a Comprehensive Human-Centred Evaluation Framework for Explainable AI
Not on the map. Why?
Abstract (arXiv)
While research on explainable AI (XAI) is booming and explanation techniques have proven promising in many application domains, standardised human-centred evaluation procedures are still missing. In addition, current evaluation procedures do not assess XAI methods holistically in the sense that they do not treat explanations' effects on humans as a complex user experience. To tackle this challenge, we propose to adapt the User-Centric Evaluation Framework used in recommender systems: we integrate explanation aspects, summarise explanation properties, indicate relations between them, and categorise metrics that measure these properties. With this comprehensive evaluation framework, we hope to contribute to the human-centred standardisation of XAI evaluation.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 53 references that are in the map. First posted on arXiv 31 Jul 2023.
Builds on
28 of 53 references.
References · 28
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
- 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
- 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
- 2018Metrics for Explainable AI: Challenges and Prospectscited by 257
- 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 138
- 2019What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Usecited by 134
- 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
- 2021Human-Centered Explainable AI (XAI): From Algorithms to User Experiencescited by 123
- 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
- 2019Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approachescited by 101
- 2020The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategiescited by 91
16 more
- 2020Causal Interpretability for Machine Learning -- Problems, Methods and Evaluationcited by 82
- 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
- 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
- 2022Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AIcited by 40
- 2021A Study of Automatic Metrics for the Evaluation of Natural Language Explanationscited by 34
- 2021Question-Driven Design Process for Explainable AI User Experiencescited by 31
- 2020Analysis of Q-learning with Adaptation and Momentum Restart for Gradient Descentcited by 26
- 2022A Survey of Explainable Graph Neural Networks: Taxonomy and Evaluation Metricscited by 26
- 2022Machine Explanations and Human Understandingcited by 20
- 2021Counterfactual Evaluation for Explainable AIcited by 15
- 2019In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processescited by 13
- 2021Roadmap of Designing Cognitive Metrics for Explainable Artificial Intelligence (XAI)cited by 13
- 2021Developing a Fidelity Evaluation Approach for Interpretable Machine Learningcited by 8
- 2022A Meta Survey of Quality Evaluation Criteria in Explanation Methodscited by 8
- 2022Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Boxcited by 7
- 2020Qualitative Investigation in Explainable Artificial Intelligence: A Bit More Insight from Social Sciencecited by 2
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.