# Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge arXiv: https://arxiv.org/abs/2308.15700 ยท HTML: https://mlsanity.com/paper/2308.15700/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Builds on 23 of its 63 references that are in the map. - First posted on arXiv 30 Aug 2023. ## Builds on (most cited first, 23 of 23) - [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020) - [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019) - [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018) - [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020) - [Learning to Complement Humans](https://mlsanity.com/paper/2005.00582/) (2020) - ["Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans](https://mlsanity.com/paper/2001.05871/) (2020) - [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021) - [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020) - [Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning](https://mlsanity.com/paper/2202.05402/) (2022) - [Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness](https://mlsanity.com/paper/2202.08821/) (2022) - [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022) - [A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms](https://mlsanity.com/paper/2206.14983/) (2022) - [Teaching Humans When To Defer to a Classifier via Exemplars](https://mlsanity.com/paper/2111.11297/) (2021) - [Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems](https://mlsanity.com/paper/2102.01196/) (2021) - [Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making](https://mlsanity.com/paper/2302.06503/) (2023) - [A Human-Centered Review of the Algorithms used within the U.S. Child Welfare System](https://mlsanity.com/paper/2003.03541/) (2020) - [A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare](https://mlsanity.com/paper/2107.03487/) (2021) - [Algorithms and Decision-Making in the Public Sector](https://mlsanity.com/paper/2106.03673/) (2021) - [Designing for human-AI complementarity in K-12 education](https://mlsanity.com/paper/2104.01266/) (2021) - [Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservables](https://mlsanity.com/paper/2207.13834/) (2022) - [On the Effect of Information Asymmetry in Human-AI Teams](https://mlsanity.com/paper/2205.01467/) (2022) - [Unpacking Invisible Work Practices, Constraints, and Latent Power Relationships in Child Welfare through Casenote Analysis](https://mlsanity.com/paper/2203.05169/) (2022) - [Heterogeneity in Algorithm-Assisted Decision-Making: A Case Study in Child Abuse Hotline Screening](https://mlsanity.com/paper/2204.05478/) (2022) ## Built on by (most cited first, 6 of 6) - [Effective Human-AI Teams via Learned Natural Language Rules and Onboarding](https://mlsanity.com/paper/2311.01007/) (2023) - [Contrastive Explanations That Anticipate Human Misconceptions Can Improve Human Decision-Making Skills](https://mlsanity.com/paper/2410.04253/) (2024) - [Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learning](https://mlsanity.com/paper/2403.05911/) (2024) - [Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information](https://mlsanity.com/paper/2401.04729/) (2024) - [Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees' Practices, Challenges, and Needs](https://mlsanity.com/paper/2411.04994/) (2024) - [Epistemic Warrant for LLM Recommendations: Characterizing the Basis for Reliance When Ground Truth Is Unavailable](https://mlsanity.com/paper/2609.04127/) (2026) ## Abstract (arXiv) A growing body of research has explored how to support humans in making better use of AI-based decision support, including via training and onboarding. Existing research has focused on decision-making tasks where it is possible to evaluate "appropriate reliance" by comparing each decision against a ground truth label that cleanly maps to both the AI's predictive target and the human decision-maker's goals. However, this assumption does not hold in many real-world settings where AI tools are deployed today (e.g., social work, criminal justice, and healthcare). In this paper, we introduce a process-oriented notion of appropriate reliance called critical use that centers the human's ability to situate AI predictions against knowledge that is uniquely available to them but unavailable to the AI model. To explore how training can support critical use, we conduct a randomized online experiment in a complex social decision-making setting: child maltreatment screening. We find that, by providing participants with accelerated, low-stakes opportunities to practice AI-assisted decision-making in this setting, novices came to exhibit patterns of disagreement with AI that resemble those of experienced workers. A qualitative examination of participants' explanations for their AI-assisted decisions revealed that they drew upon qualitative case narratives, to which the AI model did not have access, to learn when (not) to rely on AI predictions. Our findings open new questions for the study and design of training for real-world AI-assisted decision-making.