# Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory

arXiv: https://arxiv.org/abs/2205.05057 · HTML: https://mlsanity.com/paper/2205.05057/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 136 references that are in the map.
- First posted on arXiv 10 May 2022.

## Builds on (most cited first, 29 of 29)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [Expanding Explainability: Towards Social Transparency in AI systems](https://mlsanity.com/paper/2101.04719/) (2021)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- [Jury Learning: Integrating Dissenting Voices into Machine Learning Models](https://mlsanity.com/paper/2202.02950/) (2022)
- [Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences](https://mlsanity.com/paper/1712.00547/) (2017)
- [Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs](https://mlsanity.com/paper/2004.11440/) (2020)
- [Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making](https://mlsanity.com/paper/1802.01029/) (2018)
- [Contrastive Explanation: A Structural-Model Approach](https://mlsanity.com/paper/1811.03163/) (2018)
- [A causal framework for explaining the predictions of black-box sequence-to-sequence models](https://mlsanity.com/paper/1707.01943/) (2017)
- [The Who in XAI: How AI Background Shapes Perceptions of AI Explanations](https://mlsanity.com/paper/2107.13509/) (2021)
- [The Challenge of Crafting Intelligible Intelligence](https://mlsanity.com/paper/1803.04263/) (2018)
- [Interpretable Classification Models for Recidivism Prediction](https://mlsanity.com/paper/1503.07810/) (2015)
- [Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help?](https://mlsanity.com/paper/2103.08942/) (2021)
- [Towards Relatable Explainable AI with the Perceptual Process](https://mlsanity.com/paper/2112.14005/) (2021)
- [Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects](https://mlsanity.com/paper/2002.03389/) (2020)
- [From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence](https://mlsanity.com/paper/2104.13299/) (2021)
- [Simple rules for complex decisions](https://mlsanity.com/paper/1702.04690/) (2017)

## Built on by (most cited first, 30 of 32)

- [The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice](https://mlsanity.com/paper/2310.00907/) (2023)
- [Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision support](https://mlsanity.com/paper/2302.12389/) (2023)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [Seamful XAI: Operationalizing Seamful Design in Explainable AI](https://mlsanity.com/paper/2211.06753/) (2022)
- [Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience](https://mlsanity.com/paper/2302.10395/) (2023)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML](https://mlsanity.com/paper/2303.08485/) (2023)
- [AI Chatbots for Mental Health: Values and Harms from Lived Experiences of Depression](https://mlsanity.com/paper/2504.18932/) (2025)
- [Incremental XAI: Memorable Understanding of AI with Incremental Explanations](https://mlsanity.com/paper/2404.06733/) (2024)
- [Think About the Stakeholders First! Towards an Algorithmic Transparency Playbook for Regulatory Compliance](https://mlsanity.com/paper/2207.01482/) (2022)
- [A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI](https://mlsanity.com/paper/2302.05284/) (2023)
- [What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks](https://mlsanity.com/paper/2403.12730/) (2024)
- [Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents](https://mlsanity.com/paper/2606.05391/) (2026)
- [Who's Thinking? A Push for Human-Centered Evaluation of LLMs using the XAI Playbook](https://mlsanity.com/paper/2303.06223/) (2023)
- [Diagrammatization and Abduction to Improve AI Interpretability With Domain-Aligned Explanations for Medical Diagnosis](https://mlsanity.com/paper/2302.01241/) (2023)
- [Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakers](https://mlsanity.com/paper/2410.22937/) (2024)
- [Evalet: Evaluating Large Language Models through Functional Fragmentation](https://mlsanity.com/paper/2509.11206/) (2025)
- [HCC Is All You Need: Alignment-The Sensible Kind Anyway-Is Just Human-Centered Computing](https://mlsanity.com/paper/2405.03699/) (2024)
- [DLBacktrace: A Model Agnostic Explainability for any Deep Learning Models](https://mlsanity.com/paper/2411.12643/) (2024)
- [See What I Mean? CUE: A Cognitive Model of Understanding Explanations](https://mlsanity.com/paper/2506.14775/) (2025)
- [PaperTrail: A Claim-Evidence Interface for Grounding Provenance in LLM-based Scholarly Q&A](https://mlsanity.com/paper/2602.21045/) (2026)
- [Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems](https://mlsanity.com/paper/2605.16278/) (2026)
- [Explaining the ghosts: Feminist intersectional XAI and cartography as methods to account for invisible labour](https://mlsanity.com/paper/2305.03376/) (2023)
- [Applying Interdisciplinary Frameworks to Understand Algorithmic Decision-Making](https://mlsanity.com/paper/2305.16700/) (2023)
- [Alignment Is Not Enough: A Relational Framework for Moral Standing in Human-AI Interaction](https://mlsanity.com/paper/2603.00078/) (2026)
- [On the Importance of Application-Grounded Experimental Design for Evaluating Explainable ML Methods](https://mlsanity.com/paper/2206.13503/) (2022)
- [A Meta-heuristic Approach to Estimate and Explain Classifier Uncertainty](https://mlsanity.com/paper/2304.10284/) (2023)
- [Knowledge-Augmented Explainable and Interpretable Learning for Anomaly Detection and Diagnosis](https://mlsanity.com/paper/2412.00146/) (2024)
- [What Do People Want to Know About Artificial Intelligence (AI)? The Importance of Answering End-User Questions to Explain Autonomous Vehicle (AV) Decisions](https://mlsanity.com/paper/2505.06428/) (2025)
- [Visioning Human-Agentic AI Teaming: Continuity, Tension, and Future Research](https://mlsanity.com/paper/2603.04746/) (2026)

## Abstract (arXiv)

Understanding how ML models work is a prerequisite for responsibly designing, deploying, and using ML-based systems. With interpretability approaches, ML can now offer explanations for its outputs to aid human understanding. Though these approaches rely on guidelines for how humans explain things to each other, they ultimately solve for improving the artifact -- an explanation. In this paper, we propose an alternate framework for interpretability grounded in Weick's sensemaking theory, which focuses on who the explanation is intended for. Recent work has advocated for the importance of understanding stakeholders' needs -- we build on this by providing concrete properties (e.g., identity, social context, environmental cues, etc.) that shape human understanding. We use an application of sensemaking in organizations as a template for discussing design guidelines for Sensible AI, AI that factors in the nuances of human cognition when trying to explain itself.