# Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Making arXiv: https://arxiv.org/abs/2103.02071 ยท HTML: https://mlsanity.com/paper/2103.02071/ ## Key facts - Cited by 19 papers in the map (data through 14 Oct 2026). - Builds on 9 of its 35 references that are in the map. - First posted on arXiv 2 Mar 2021. ## Builds on (most cited first, 9 of 9) - [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017) - [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017) - [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016) - [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019) - [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018) - [explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning](https://mlsanity.com/paper/1908.00087/) (2019) - [RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records](https://mlsanity.com/paper/1805.10724/) (2018) - [ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models](https://mlsanity.com/paper/1704.01942/) (2017) - [Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models](https://mlsanity.com/paper/1808.00196/) (2018) ## Built on by (most cited first, 19 of 19) - [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022) - [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023) - [On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs](https://mlsanity.com/paper/2407.19200/) (2024) - [LLMs for XAI: Future Directions for Explaining Explanations](https://mlsanity.com/paper/2405.06064/) (2024) - [Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic review](https://mlsanity.com/paper/2310.19778/) (2023) - [Designing Interpretable ML System to Enhance Trust in Healthcare: A Systematic Review to Proposed Responsible Clinician-AI-Collaboration Framework](https://mlsanity.com/paper/2311.11055/) (2023) - [Towards an Understanding and Explanation for Mixed-Initiative Artificial Scientific Text Detection](https://mlsanity.com/paper/2304.05011/) (2023) - [Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use](https://mlsanity.com/paper/2405.12458/) (2024) - [Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks](https://mlsanity.com/paper/2502.10526/) (2025) - [Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts](https://mlsanity.com/paper/2205.05749/) (2022) - [The Need for Interpretable Features: Motivation and Taxonomy](https://mlsanity.com/paper/2202.11748/) (2022) - [Explain To Decide: A Human-Centric Review on the Role of Explainable Artificial Intelligence in AI-assisted Decision Making](https://mlsanity.com/paper/2312.11507/) (2023) - [Machine Learning in Transaction Monitoring: The Prospect of xAI](https://mlsanity.com/paper/2210.07648/) (2022) - [Underspecified Human Decision Experiments Considered Harmful](https://mlsanity.com/paper/2401.15106/) (2024) - [Lessons from Usable ML Deployments and Application to Wind Turbine Monitoring](https://mlsanity.com/paper/2312.02859/) (2023) - [Pyreal: A Framework for Interpretable ML Explanations](https://mlsanity.com/paper/2312.13084/) (2023) - [Explanation User Interfaces: A Systematic Literature Review](https://mlsanity.com/paper/2505.20085/) (2025) - [Restricted Receptive Fields for Face Verification](https://mlsanity.com/paper/2510.10753/) (2025) - [Intelligent Reasoning Cues: A Framework and Case Study of the Roles of AI Information in Complex Decisions](https://mlsanity.com/paper/2602.00259/) (2026) ## Abstract (arXiv) Machine learning (ML) is being applied to a diverse and ever-growing set of domains. In many cases, domain experts - who often have no expertise in ML or data science - are asked to use ML predictions to make high-stakes decisions. Multiple ML usability challenges can appear as result, such as lack of user trust in the model, inability to reconcile human-ML disagreement, and ethical concerns about oversimplification of complex problems to a single algorithm output. In this paper, we investigate the ML usability challenges that present in the domain of child welfare screening through a series of collaborations with child welfare screeners. Following the iterative design process between the ML scientists, visualization researchers, and domain experts (child screeners), we first identified four key ML challenges and honed in on one promising explainable ML technique to address them (local factor contributions). Then we implemented and evaluated our visual analytics tool, Sibyl, to increase the interpretability and interactivity of local factor contributions. The effectiveness of our tool is demonstrated by two formal user studies with 12 non-expert participants and 13 expert participants respectively. Valuable feedback was collected, from which we composed a list of design implications as a useful guideline for researchers who aim to develop an interpretable and interactive visualization tool for ML prediction models deployed for child welfare screeners and other similar domain experts.