Requirements for Explainability and Acceptance of Artificial Intelligence in Collaborative Work
Not on the map. Why?
Abstract (arXiv)
The increasing prevalence of Artificial Intelligence (AI) in safety-critical contexts such as air-traffic control leads to systems that are practical and efficient, and to some extent explainable to humans to be trusted and accepted. The present structured literature analysis examines n = 236 articles on the requirements for the explainability and acceptance of AI. Results include a comprehensive review of n = 48 articles on information people need to perceive an AI as explainable, the information needed to accept an AI, and representation and interaction methods promoting trust in an AI. Results indicate that the two main groups of users are developers who require information about the internal operations of the model and end users who require information about AI results or behavior. Users' information needs vary in specificity, complexity, and urgency and must consider context, domain knowledge, and the user's cognitive resources. The acceptance of AI systems depends on information about the system's functions and performance, privacy and ethical considerations, as well as goal-supporting information tailored to individual preferences and information to establish trust in the system. Information about the system's limitations and potential failures can increase acceptance and trust. Trusted interaction methods are human-like, including natural language, speech, text, and visual representations such as graphs, charts, and animations. Our results have significant implications for future human-centric AI systems being developed. Thus, they are suitable as input for further application-specific investigations of user needs.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 25 of its 124 references that are in the map. First posted on arXiv 27 Jun 2023.
Builds on
25 of 124 references.
References · 25
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
- 2018Sanity Checks for Saliency Mapscited by 1,156
- 2018Model Cards for Model Reportingcited by 1,080
- 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
- 2017Interpretation of Neural Networks is Fragilecited by 444
- 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
- 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
- 2017The (Un)reliability of saliency methodscited by 333
- 2021Skillful Precipitation Nowcasting using Deep Generative Models of Radarcited by 311
- 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
13 more
- 2020A Survey on the Explainability of Supervised Machine Learningcited by 227
- 2021A Survey of Human-in-the-loop for Machine Learningcited by 215
- 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
- 2021Why AI is Harder Than We Thinkcited by 41
- 2019Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulationcited by 35
- 2020Semantics of the Black-Box: Can knowledge graphs help make deep learning systems more interpretable and explainable?cited by 31
- 2021EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: the MonuMAI cultural heritage use casecited by 30
- 2020Reviewing the Need for Explainable Artificial Intelligence (xAI)cited by 29
- 2022Trust in AI and Its Role in the Acceptance of AI Technologiescited by 29
- 2021On the Relation of Trust and Explainability: Why to Engineer for Trustworthinesscited by 16
- 2018Explainable Software Analyticscited by 15
- 2017Explaining Trained Neural Networks with Semantic Web Technologies: First Stepscited by 13
- 2014Towards the Safety of Human-in-the-Loop Robotics: Challenges and Opportunities for Safety Assurance of Robotic Co-Workerscited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.