# Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support

arXiv: https://arxiv.org/abs/2204.02310 · HTML: https://mlsanity.com/paper/2204.02310/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 71 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 22 of 22)

- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- [Building machines that adapt and compute like brains](https://mlsanity.com/paper/1711.04203/) (2017)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020)
- [Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making](https://mlsanity.com/paper/1802.01029/) (2018)
- [Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes](https://mlsanity.com/paper/1904.09612/) (2019)
- [Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors](https://mlsanity.com/paper/2105.02980/) (2021)
- [The Flaws of Policies Requiring Human Oversight of Government Algorithms](https://mlsanity.com/paper/2109.05067/) (2021)
- [Teaching Humans When To Defer to a Classifier via Exemplars](https://mlsanity.com/paper/2111.11297/) (2021)
- [Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems](https://mlsanity.com/paper/2102.01196/) (2021)
- [A Human-Centered Review of the Algorithms used within the U.S. Child Welfare System](https://mlsanity.com/paper/2003.03541/) (2020)
- [A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare](https://mlsanity.com/paper/2107.03487/) (2021)
- [Algorithms and Decision-Making in the Public Sector](https://mlsanity.com/paper/2106.03673/) (2021)
- [Investigating Human + Machine Complementarity for Recidivism Predictions](https://mlsanity.com/paper/1808.09123/) (2018)
- [Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Making](https://mlsanity.com/paper/2103.02071/) (2021)
- [Designing for human-AI complementarity in K-12 education](https://mlsanity.com/paper/2104.01266/) (2021)
- [Towards A Process Model for Co-Creating AI Experiences](https://mlsanity.com/paper/2104.07595/) (2021)
- [Leveraging Expert Consistency to Improve Algorithmic Decision Support](https://mlsanity.com/paper/2101.09648/) (2021)
- [Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems](https://mlsanity.com/paper/2001.04879/) (2020)

## Built on by (most cited first, 30 of 39)

- [Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations](https://mlsanity.com/paper/2301.07255/) (2023)
- [Improving Human-AI Collaboration With Descriptions of AI Behavior](https://mlsanity.com/paper/2301.06937/) (2023)
- [A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms](https://mlsanity.com/paper/2206.14983/) (2022)
- [Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services](https://mlsanity.com/paper/2303.09743/) (2023)
- [Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making](https://mlsanity.com/paper/2302.06503/) (2023)
- [A Taxonomy of Human and ML Strengths in Decision-Making to Investigate Human-ML Complementarity](https://mlsanity.com/paper/2204.10806/) (2022)
- [Effective Human-AI Teams via Learned Natural Language Rules and Onboarding](https://mlsanity.com/paper/2311.01007/) (2023)
- [Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservables](https://mlsanity.com/paper/2207.13834/) (2022)
- [Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making](https://mlsanity.com/paper/2308.04375/) (2023)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders](https://mlsanity.com/paper/2205.08928/) (2022)
- [Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making](https://mlsanity.com/paper/2302.08807/) (2023)
- [Counterfactual Prediction Under Outcome Measurement Error](https://mlsanity.com/paper/2302.11121/) (2023)
- [AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development](https://mlsanity.com/paper/2502.18682/) (2025)
- [LLMs and Childhood Safety: Identifying Risks and Proposing a Protection Framework for Safe Child-LLM Interaction](https://mlsanity.com/paper/2502.11242/) (2025)
- [Safe-Child-LLM: A Developmental Benchmark for Evaluating LLM Safety in Child-LLM Interactions](https://mlsanity.com/paper/2506.13510/) (2025)
- [Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge](https://mlsanity.com/paper/2308.15700/) (2023)
- [Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use](https://mlsanity.com/paper/2405.12458/) (2024)
- [Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks](https://mlsanity.com/paper/2502.10526/) (2025)
- [Discretionary Trees: Understanding Street-Level Bureaucracy via Machine Learning](https://mlsanity.com/paper/2312.10694/) (2023)
- [Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information](https://mlsanity.com/paper/2401.04729/) (2024)
- [Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making](https://mlsanity.com/paper/2505.18066/) (2025)
- [(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court](https://mlsanity.com/paper/2403.13004/) (2024)
- [Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process](https://mlsanity.com/paper/2406.08959/) (2024)
- [Explainable and Human-Grounded AI for Decision Support Systems: The Theory of Epistemic Quasi-Partnerships](https://mlsanity.com/paper/2409.14839/) (2024)
- [Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakers](https://mlsanity.com/paper/2410.22937/) (2024)
- [Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees' Practices, Challenges, and Needs](https://mlsanity.com/paper/2411.04994/) (2024)
- [Street-Level AI: Are Large Language Models Ready for Real-World Judgments?](https://mlsanity.com/paper/2508.08193/) (2025)
- [Talk2Care: Facilitating Asynchronous Patient-Provider Communication with Large-Language-Model](https://mlsanity.com/paper/2309.09357/) (2023)
- [Measurement as Bricolage: Examining How Data Scientists Construct Target Variables for Predictive Modeling Tasks](https://mlsanity.com/paper/2507.02819/) (2025)

## Abstract (arXiv)

AI-based decision support tools (ADS) are increasingly used to augment human decision-making in high-stakes, social contexts. As public sector agencies begin to adopt ADS, it is critical that we understand workers' experiences with these systems in practice. In this paper, we present findings from a series of interviews and contextual inquiries at a child welfare agency, to understand how they currently make AI-assisted child maltreatment screening decisions. Overall, we observe how workers' reliance upon the ADS is guided by (1) their knowledge of rich, contextual information beyond what the AI model captures, (2) their beliefs about the ADS's capabilities and limitations relative to their own, (3) organizational pressures and incentives around the use of the ADS, and (4) awareness of misalignments between algorithmic predictions and their own decision-making objectives. Drawing upon these findings, we discuss design implications towards supporting more effective human-AI decision-making.