# Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders

arXiv: https://arxiv.org/abs/2205.08928 · HTML: https://mlsanity.com/paper/2205.08928/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 146 references that are in the map.
- First posted on arXiv 18 May 2022.

## Builds on (most cited first, 16 of 16)

- [Participation is not a Design Fix for Machine Learning](https://mlsanity.com/paper/2007.02423/) (2020)
- [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020)
- [Counterfactual Risk Assessments, Evaluation, and Fairness](https://mlsanity.com/paper/1909.00066/) (2019)
- [A Voting-Based System for Ethical Decision Making](https://mlsanity.com/paper/1709.06692/) (2017)
- [Metric Learning for Individual Fairness](https://mlsanity.com/paper/1906.00250/) (2019)
- [The Flaws of Policies Requiring Human Oversight of Government Algorithms](https://mlsanity.com/paper/2109.05067/) (2021)
- [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022)
- [Metric-Free Individual Fairness in Online Learning](https://mlsanity.com/paper/2002.05474/) (2020)
- [Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems](https://mlsanity.com/paper/2102.01196/) (2021)
- [Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir"](https://mlsanity.com/paper/2111.01122/) (2021)
- [A Human-Centered Review of the Algorithms used within the U.S. Child Welfare System](https://mlsanity.com/paper/2003.03541/) (2020)
- [A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare](https://mlsanity.com/paper/2107.03487/) (2021)
- [ORES: Lowering Barriers with Participatory Machine Learning in Wikipedia](https://mlsanity.com/paper/1909.05189/) (2019)
- [Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems](https://mlsanity.com/paper/2001.04879/) (2020)
- [What If I Don't Like Any Of The Choices? The Limits of Preference Elicitation for Participatory Algorithm Design](https://mlsanity.com/paper/2007.06718/) (2020)
- [How to Train a (Bad) Algorithmic Caseworker: A Quantitative Deconstruction of Risk Assessments in Child-Welfare](https://mlsanity.com/paper/2203.05695/) (2022)

## Built on by (most cited first, 15 of 15)

- [Harms from Increasingly Agentic Algorithmic Systems](https://mlsanity.com/paper/2302.10329/) (2023)
- [The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice](https://mlsanity.com/paper/2310.00907/) (2023)
- [Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services](https://mlsanity.com/paper/2303.09743/) (2023)
- [LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions](https://mlsanity.com/paper/2406.08824/) (2024)
- [Lazy Data Practices Harm Fairness Research](https://mlsanity.com/paper/2404.17293/) (2024)
- [AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development](https://mlsanity.com/paper/2502.18682/) (2025)
- [A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI](https://mlsanity.com/paper/2302.05284/) (2023)
- [Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use](https://mlsanity.com/paper/2405.12458/) (2024)
- [Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making](https://mlsanity.com/paper/2505.18066/) (2025)
- [Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees' Practices, Challenges, and Needs](https://mlsanity.com/paper/2411.04994/) (2024)
- [Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agency](https://mlsanity.com/paper/2506.07281/) (2025)
- [The AI-DEC: A Card-based Design Method for User-centered AI Explanations](https://mlsanity.com/paper/2405.16711/) (2024)
- [Beyond Predictive Algorithms in Child Welfare](https://mlsanity.com/paper/2403.05573/) (2024)
- [Toward AI Matching Policies in Homeless Services: A Qualitative Study with Policymakers](https://mlsanity.com/paper/2508.07129/) (2025)
- [RuleEdit: Failure-Guided Human-AI Model Editing with Prospective Impact Preview](https://mlsanity.com/paper/2606.00011/) (2026)

## Abstract (arXiv)

Child welfare agencies across the United States are turning to data-driven predictive technologies (commonly called predictive analytics) which use government administrative data to assist workers' decision-making. While some prior work has explored impacted stakeholders' concerns with current uses of data-driven predictive risk models (PRMs), less work has asked stakeholders whether such tools ought to be used in the first place. In this work, we conducted a set of seven design workshops with 35 stakeholders who have been impacted by the child welfare system or who work in it to understand their beliefs and concerns around PRMs, and to engage them in imagining new uses of data and technologies in the child welfare system. We found that participants worried current PRMs perpetuate or exacerbate existing problems in child welfare. Participants suggested new ways to use data and data-driven tools to better support impacted communities and suggested paths to mitigate possible harms of these tools. Participants also suggested low-tech or no-tech alternatives to PRMs to address problems in child welfare. Our study sheds light on how researchers and designers can work in solidarity with impacted communities, possibly to circumvent or oppose child welfare agencies.