# Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI

arXiv: https://arxiv.org/abs/2302.00799 · HTML: https://mlsanity.com/paper/2302.00799/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 171 references that are in the map.
- First posted on arXiv 1 Feb 2023.

## Builds on (most cited first, 30 of 48)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Understanding Neural Networks Through Deep Visualization](https://mlsanity.com/paper/1506.06579/) (2015)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy](https://mlsanity.com/paper/2002.04087/) (2020)
- [Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI](https://mlsanity.com/paper/2204.01075/) (2022)
- [Local Rule-Based Explanations of Black Box Decision Systems](https://mlsanity.com/paper/1805.10820/) (2018)
- [One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques](https://mlsanity.com/paper/1909.03012/) (2019)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [Explaining Explanations in AI](https://mlsanity.com/paper/1811.01439/) (2018)
- [What do we need to build explainable AI systems for the medical domain?](https://mlsanity.com/paper/1712.09923/) (2017)
- [Expanding Explainability: Towards Social Transparency in AI systems](https://mlsanity.com/paper/2101.04719/) (2021)
- [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](https://mlsanity.com/paper/2110.10790/) (2021)
- [Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches](https://mlsanity.com/paper/1912.05100/) (2019)
- [Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence](https://mlsanity.com/paper/2007.04068/) (2020)
- [Human-centered Explainable AI: Towards a Reflective Sociotechnical Approach](https://mlsanity.com/paper/2002.01092/) (2020)
- [Interpreting CNNs via Decision Trees](https://mlsanity.com/paper/1802.00121/) (2018)
- ["Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans](https://mlsanity.com/paper/2001.05871/) (2020)
- [Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Study](https://mlsanity.com/paper/2002.00772/) (2020)
- [Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions](https://mlsanity.com/paper/1901.03729/) (2019)
- [The age of secrecy and unfairness in recidivism prediction](https://mlsanity.com/paper/1811.00731/) (2018)

## Built on by (most cited first, 30 of 33)

- [Perceptions of Sentient AI and Other Digital Minds: Evidence from the AI, Morality, and Sentience (AIMS) Survey](https://mlsanity.com/paper/2407.08867/) (2024)
- [Artificial Intelligence Should Genuinely Support Clinical Reasoning and Decision Making To Bridge the Translational Gap](https://mlsanity.com/paper/2506.05030/) (2025)
- [AI Chatbots for Mental Health: Values and Harms from Lived Experiences of Depression](https://mlsanity.com/paper/2504.18932/) (2025)
- [Mapping Caregiver Needs to AI Chatbot Design: Strengths and Gaps in Mental Health Support for Alzheimer's and Dementia Caregivers](https://mlsanity.com/paper/2506.15047/) (2025)
- [Trustless Autonomy: Understanding Motivations, Benefits, and Governance Dilemmas in Self-Sovereign Decentralized AI Agents](https://mlsanity.com/paper/2505.09757/) (2025)
- [Mind the Gap! Bridging Explainable Artificial Intelligence and Human Understanding with Luhmann's Functional Theory of Communication](https://mlsanity.com/paper/2302.03460/) (2023)
- [Balancing Caregiving and Self-Care: Exploring Mental Health Needs of Alzheimer's and Dementia Caregivers](https://mlsanity.com/paper/2506.14196/) (2025)
- [A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI](https://mlsanity.com/paper/2302.05284/) (2023)
- [What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks](https://mlsanity.com/paper/2403.12730/) (2024)
- [Linguistic Comparison of AI- and Human-Written Responses to Online Mental Health Queries](https://mlsanity.com/paper/2504.09271/) (2025)
- [Can we Debias Social Stereotypes in AI-Generated Images? Examining Text-to-Image Outputs and User Perceptions](https://mlsanity.com/paper/2505.20692/) (2025)
- [From Future of Work to Future of Workers: Addressing Asymptomatic AI Harms for Dignified Human-AI Interaction](https://mlsanity.com/paper/2601.21920/) (2026)
- [An Information Bottleneck Characterization of the Understanding-Workload Tradeoff](https://mlsanity.com/paper/2310.07802/) (2023)
- [The Value-Sensitive Conversational Agent Co-Design Framework](https://mlsanity.com/paper/2310.11848/) (2023)
- [Information That Matters: Exploring Information Needs of People Affected by Algorithmic Decisions](https://mlsanity.com/paper/2401.13324/) (2024)
- [Human-Centric eXplainable AI in Education](https://mlsanity.com/paper/2410.19822/) (2024)
- [AI Thinking: A framework for rethinking artificial intelligence in practice](https://mlsanity.com/paper/2409.12922/) (2024)
- [A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Models](https://mlsanity.com/paper/2504.02793/) (2025)
- [InterVLS: Interactive Model Understanding and Improvement with Vision-Language Surrogates](https://mlsanity.com/paper/2311.03547/) (2023)
- ["It is there, and you need it, so why do you not use it?" Achieving better adoption of AI systems by domain experts, in the case study of natural science research](https://mlsanity.com/paper/2403.16895/) (2024)
- ["Impressively Scary:" Exploring User Perceptions and Reactions to Unraveling Machine Learning Models in Social Media Applications](https://mlsanity.com/paper/2503.03927/) (2025)
- [Designing KRIYA: An AI Companion for Wellbeing Self-Reflection](https://mlsanity.com/paper/2601.14589/) (2026)
- [A Checklist for Trustworthy, Safe, and User-Friendly Mental Health Chatbots](https://mlsanity.com/paper/2601.15412/) (2026)
- [What is Safety? Corporate Discourse, Power, and the Politics of Generative AI Safety](https://mlsanity.com/paper/2602.06981/) (2026)
- [Empowering 9-1-1 Calltaking Training with Generative AI: Experiences and Lessons Learned](https://mlsanity.com/paper/2602.13241/) (2026)
- [Explainable AI Reloaded: Challenging the XAI Status Quo in the Era of Large Language Models](https://mlsanity.com/paper/2408.05345/) (2024)
- [Human-centered explanation does not fit all: The interplay of sociotechnical, cognitive, and individual factors in the effect AI explanations in algorithmic decision-making](https://mlsanity.com/paper/2502.12354/) (2025)
- [Beyond Technocratic XAI: The Who, What & How in Explanation Design](https://mlsanity.com/paper/2508.09231/) (2025)
- [Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Study](https://mlsanity.com/paper/2512.15791/) (2025)
- [Emergent, not Immanent: A Baradian Reading of Explainable AI](https://mlsanity.com/paper/2601.15029/) (2026)

## Abstract (arXiv)

Explainable AI (XAI) systems are sociotechnical in nature; thus, they are subject to the sociotechnical gap--divide between the technical affordances and the social needs. However, charting this gap is challenging. In the context of XAI, we argue that charting the gap improves our problem understanding, which can reflexively provide actionable insights to improve explainability. Utilizing two case studies in distinct domains, we empirically derive a framework that facilitates systematic charting of the sociotechnical gap by connecting AI guidelines in the context of XAI and elucidating how to use them to address the gap. We apply the framework to a third case in a new domain, showcasing its affordances. Finally, we discuss conceptual implications of the framework, share practical considerations in its operationalization, and offer guidance on transferring it to new contexts. By making conceptual and practical contributions to understanding the sociotechnical gap in XAI, the framework expands the XAI design space.