Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders
Not on the map. Why?
Abstract (arXiv)
Child welfare agencies across the United States are turning to data-driven predictive technologies (commonly called predictive analytics) which use government administrative data to assist workers' decision-making. While some prior work has explored impacted stakeholders' concerns with current uses of data-driven predictive risk models (PRMs), less work has asked stakeholders whether such tools ought to be used in the first place. In this work, we conducted a set of seven design workshops with 35 stakeholders who have been impacted by the child welfare system or who work in it to understand their beliefs and concerns around PRMs, and to engage them in imagining new uses of data and technologies in the child welfare system. We found that participants worried current PRMs perpetuate or exacerbate existing problems in child welfare. Participants suggested new ways to use data and data-driven tools to better support impacted communities and suggested paths to mitigate possible harms of these tools. Participants also suggested low-tech or no-tech alternatives to PRMs to address problems in child welfare. Our study sheds light on how researchers and designers can work in solidarity with impacted communities, possibly to circumvent or oppose child welfare agencies.
Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 16 of its 146 references that are in the map. First posted on arXiv 18 May 2022.
Builds on
16 of 146 references.
References · 16
- 2020Participation is not a Design Fix for Machine Learningcited by 110
- 2020A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scorescited by 74
- 2019Counterfactual Risk Assessments, Evaluation, and Fairnesscited by 59
- 2017A Voting-Based System for Ethical Decision Makingcited by 55
- 2019Metric Learning for Individual Fairnesscited by 51
- 2021The Flaws of Policies Requiring Human Oversight of Government Algorithmscited by 44
- 2022Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Supportcited by 37
- 2020Metric-Free Individual Fairness in Online Learningcited by 30
- 2021Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systemscited by 27
- 2021Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir"cited by 26
- 2020A Human-Centered Review of the Algorithms used within the U.S. Child Welfare Systemcited by 23
- 2021A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfarecited by 23
4 more
- 2019ORES: Lowering Barriers with Participatory Machine Learning in Wikipediacited by 16
- 2020Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systemscited by 16
- 2020What If I Don't Like Any Of The Choices? The Limits of Preference Elicitation for Participatory Algorithm Designcited by 14
- 2022How to Train a (Bad) Algorithmic Caseworker: A Quantitative Deconstruction of Risk Assessments in Child-Welfarecited by 4
Built on by
Citing papers per year: 2023: 4; 2024: 6; 2025: 4; 2026: 1.
15 papers.
Sort:
By area:
- 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
- 2023The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practicecited by 86
- 2023Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Servicescited by 29
- 2024LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actionscited by 21
- 2024Lazy Data Practices Harm Fairness Researchcited by 10
- 2025AI Mismatches: Identifying Potential Algorithmic Harms Before AI Developmentcited by 10
- 2024Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Usecited by 6
- 2023A Systematic Literature Review of Human-Centered, Ethical, and Responsible AIcited by 5
- 2025Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Makingcited by 4
- 2024Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees' Practices, Challenges, and Needscited by 3
- 2025Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agencycited by 3
- 2024The AI-DEC: A Card-based Design Method for User-centered AI Explanationscited by 2
- 2026RuleEdit: Failure-Guided Human-AI Model Editing with Prospective Impact Preview12 Apr 2026
- 2025Toward AI Matching Policies in Homeless Services: A Qualitative Study with Policymakers10 Aug 2025
- 2025Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agency8 Jun 2025
- 2025Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making23 May 2025
- 2025AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development25 Feb 2025
- 2024Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees' Practices, Challenges, and Needs7 Nov 2024
- 2024LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions13 Jun 2024
- 2024The AI-DEC: A Card-based Design Method for User-centered AI Explanations26 May 2024
- 2024Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use21 May 2024
- 2024Lazy Data Practices Harm Fairness Research26 Apr 2024
- 2024Beyond Predictive Algorithms in Child Welfare26 Feb 2024
- 2023The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice2 Oct 2023
- 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
- 2023Harms from Increasingly Agentic Algorithmic Systems20 Feb 2023
- 2023The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practicecited by 86
- 2023Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Servicescited by 29
- 2024LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actionscited by 21
- 2024Lazy Data Practices Harm Fairness Researchcited by 10
- 2025AI Mismatches: Identifying Potential Algorithmic Harms Before AI Developmentcited by 10
- 2024Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Usecited by 6
- 2023A Systematic Literature Review of Human-Centered, Ethical, and Responsible AIcited by 5
- 2025Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Makingcited by 4
- 2024Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees' Practices, Challenges, and Needscited by 3
- 2025Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agencycited by 3
- 2024The AI-DEC: A Card-based Design Method for User-centered AI Explanationscited by 2
- 2024Beyond Predictive Algorithms in Child Welfarecited by 0
12 most cited of 14 All 14 (JSON)
- 2026RuleEdit: Failure-Guided Human-AI Model Editing with Prospective Impact Preview12 Apr 2026
- 2025Toward AI Matching Policies in Homeless Services: A Qualitative Study with Policymakers10 Aug 2025
- 2025Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agency8 Jun 2025
- 2025Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making23 May 2025
- 2025AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development25 Feb 2025
- 2024Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees' Practices, Challenges, and Needs7 Nov 2024
- 2024LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions13 Jun 2024
- 2024The AI-DEC: A Card-based Design Method for User-centered AI Explanations26 May 2024
- 2024Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use21 May 2024
- 2024Lazy Data Practices Harm Fairness Research26 Apr 2024
- 2024Beyond Predictive Algorithms in Child Welfare26 Feb 2024
- 2023The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice2 Oct 2023
12 newest of 14 All 14 (JSON)