mlsanityData to 7 Oct 2026

On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI System

arXiv 2112.01016 ↗First posted 2 Dec 2021cs.HC cs.AI cs.CYcited by 5

Not on the map. Why?

Abstract (arXiv)

Explainable AI (XAI) research has been booming, but the question "To whom are we making AI explainable?" is yet to gain sufficient attention. Not much of XAI is comprehensible to non-AI experts, who nonetheless, are the primary audience and major stakeholders of deployed AI systems in practice. The gap is glaring: what is considered "explained" to AI-experts versus non-experts are very different in practical scenarios. Hence, this gap produced two distinct cultures of expectations, goals, and forms of XAI in real-life AI deployments. We advocate that it is critical to develop XAI methods for non-technical audiences. We then present a real-life case study, where AI experts provided non-technical explanations of AI decisions to non-technical stakeholders, and completed a successful deployment in a highly regulated industry. We then synthesize lessons learned from the case, and share a list of suggestions for AI experts to consider when explaining AI decisions to non-technical stakeholders.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 14 of its 55 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

14 of 55 references.

References · 14

  1. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  2. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  3. 2016The Mythos of Model Interpretabilitycited by 716
  4. 2018Actionable Recourse in Linear Classificationcited by 327
  5. 2019Explainable Machine Learning in Deploymentcited by 210
  6. 2018Explaining Explanations in AIcited by 136
  7. 2017What do we need to build explainable AI systems for the medical domain?cited by 127
  8. 2019Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approachescited by 101
  9. 2017Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciencescited by 84
  10. 2018How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanationcited by 71
  11. 2018Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systemscited by 47
  12. 2019Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creatorscited by 31
2 more
  1. 2018A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learningcited by 21
  2. 2019Towards a Characterization of Explainable Systemscited by 4

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2024LLMs for XAI: Future Directions for Explaining Explanationscited by 19
  2. 2024Explingo: Explaining AI Predictions using Large Language Modelscited by 6
  3. 2023Lessons from Usable ML Deployments and Application to Wind Turbine Monitoringcited by 0
  4. 2023Pyreal: A Framework for Interpretable ML Explanationscited by 0
  5. 2025Feature-Guided Neighbor Selection for Non-Expert Evaluation of Model Predictionscited by 0