mlsanityData to 7 Oct 2026

Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations

arXiv 1912.09024 ↗First posted 19 Dec 2019cs.AIcited by 63

Not on the map. Why?

Abstract (arXiv)

Recent success in Artificial Intelligence (AI) and Machine Learning (ML) allow problem solving automatically without any human intervention. Autonomous approaches can be very convenient. However, in certain domains, e.g., in the medical domain, it is necessary to enable a domain expert to understand, why an algorithm came up with a certain result. Consequently, the field of Explainable AI (xAI) rapidly gained interest worldwide in various domains, particularly in medicine. Explainable AI studies transparency and traceability of opaque AI/ML and there are already a huge variety of methods. For example with layer-wise relevance propagation relevant parts of inputs to, and representations in, a neural network which caused a result, can be highlighted. This is a first important step to ensure that end users, e.g., medical professionals, assume responsibility for decision making with AI/ML and of interest to professionals and regulators. Interactive ML adds the component of human expertise to AI/ML processes by enabling them to re-enact and retrace AI/ML results, e.g. let them check it for plausibility. This requires new human-AI interfaces for explainable AI. In order to build effective and efficient interactive human-AI interfaces we have to deal with the question of how to evaluate the quality of explanations given by an explainable AI system. In this paper we introduce our System Causability Scale (SCS) to measure the quality of explanations. It is based on our notion of Causability (Holzinger et al., 2019) combined with concepts adapted from a widely accepted usability scale.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 13; 2023: 12; 2024: 10; 2025: 8; 2026: 7.

63 papers.

Sort:

By area:

  1. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  2. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 84
  3. 2021Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methodscited by 83
  4. 2023Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulationcited by 60
  5. 2021Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applicationscited by 59
  6. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  7. 2025Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistantcited by 26
  8. 2022Explainable AI for clinical and remote health applications: a survey on tabular and time series datacited by 24
  9. 2024Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlookscited by 18
  10. 2021Deep ROC Analysis and AUC as Balanced Average Accuracy to Improve Model Selection, Understanding and Interpretationcited by 17
  11. 2021An Objective Metric for Explainable AI: How and Why to Estimate the Degree of Explainabilitycited by 15
  12. 2023A Review of the Role of Causality in Developing Trustworthy AI Systemscited by 14
38 more
  1. 2021FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imagingcited by 13
  2. 2021Resisting Out-of-Distribution Data Problem in Perturbation of XAIcited by 12
  3. 2022Keep Your Friends Close and Your Counterfactuals Closer: Improved Learning From Closest Rather Than Plausible Counterfactual Explanations in an Abstract Settingcited by 11
  4. 2022Greybox XAI: a Neural-Symbolic learning framework to produce interpretable predictions for image classificationcited by 11
  5. 2022Explainable AI for Bioinformatics: Methods, Tools, and Applicationscited by 11
  6. 2023Can We Trust Explainable AI Methods on ASR? An Evaluation on Phoneme Recognitioncited by 10
  7. 2023Decoding visual brain representations from electroencephalography through Knowledge Distillation and latent diffusion modelscited by 10
  8. 2022Let's Go to the Alien Zoo: Introducing an Experimental Framework to Study Usability of Counterfactual Explanations for Machine Learningcited by 9
  9. 2021A Survey on Methods and Metrics for the Assessment of Explainability under the Proposed AI Actcited by 8
  10. 2022A Meta Survey of Quality Evaluation Criteria in Explanation Methodscited by 8
  11. 2024How well can a large language model explain business processes as perceived by users?cited by 8
  12. 2022Think About the Stakeholders First! Towards an Algorithmic Transparency Playbook for Regulatory Compliancecited by 7
  13. 2024What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blockscited by 6
  14. 2021Applications of Explainable AI for 6G: Technical Aspects, Use Cases, and Research Challengescited by 5
  15. 2022Segmentation-guided Domain Adaptation and Data Harmonization of Multi-device Retinal Optical Coherence Tomography using Cycle-Consistent Generative Adversarial Networkscited by 5
  16. 2023A New Perspective on Evaluation Methods for Explainable Artificial Intelligence (XAI)cited by 5
  17. 2026NOVA: NOise-aware Verbal Confidence CAlibration for Robust Large Language Models in RAG Systemscited by 5
  18. 2021Explanatory Pluralism in Explainable AIcited by 4
  19. 2022AI, Opacity, and Personal Autonomycited by 4
  20. 2021Optimising Knee Injury Detection with Spatial Attention and Validating Localisation Abilitycited by 3
  21. 2023For Better or Worse: The Impact of Counterfactual Explanations' Directionality on User Behavior in xAIcited by 3
  22. 2023Towards Interpretability in Audio and Visual Affective Machine Learning: A Reviewcited by 3
  23. 2025Towards Balancing Preference and Performance through Adaptive Personalized Explainabilitycited by 3
  24. 2025A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoptioncited by 3
  25. 2022Grounding Explainability Within the Context of Global South in XAIcited by 2
  26. 2023On the Multiple Roles of Ontologies in Explainable AIcited by 2
  27. 2024CoProNN: Concept-based Prototypical Nearest Neighbors for Explaining Vision Modelscited by 2
  28. 2024XEQ Scale for Evaluating XAI Experience Qualitycited by 2
  29. 2025Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainabilitycited by 2
  30. 2025Explainable AI Systems Must Be Contestable: Here's How to Make It Happencited by 2
  31. 2021Explainable AI (XAI) for PHM of Industrial Asset: A State-of-The-Art, PRISMA-Compliant Systematic Reviewcited by 1
  32. 2022Context-dependent Explainability and Contestability for Trustworthy Medical Artificial Intelligence: Misclassification Identification of Morbidity Recognition Models in Preterm Infantscited by 1
  33. 2023Distance-Aware eXplanation Based Learningcited by 1
  34. 2023On the Definition of Appropriate Trust and the Tools that Come with itcited by 1
  35. 2023How much informative is your XAI? A decision-making assessment task to objectively measure the goodness of explanationscited by 1
  36. 2024XAI-CF -- Examining the Role of Explainable Artificial Intelligence in Cyber Forensicscited by 1
  37. 2024Exploring Commonalities in Explanation Frameworks: A Multi-Domain Survey Analysiscited by 1
  38. 2025Extending Decision Predicate Graphs for Comprehensive Explanation of Isolation Forestcited by 1

50 most cited of 63 All 63 (JSON)