# Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems

arXiv: https://arxiv.org/abs/2102.01196 · HTML: https://mlsanity.com/paper/2102.01196/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2021.

## Built on by (most cited first, 27 of 27)

- [The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice](https://mlsanity.com/paper/2310.00907/) (2023)
- [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022)
- [Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services](https://mlsanity.com/paper/2303.09743/) (2023)
- [Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics](https://mlsanity.com/paper/2202.08792/) (2022)
- [Fairness Perceptions of Algorithmic Decision-Making: A Systematic Review of the Empirical Literature](https://mlsanity.com/paper/2103.12016/) (2021)
- [Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders](https://mlsanity.com/paper/2205.08928/) (2022)
- [Towards Responsible AI: A Design Space Exploration of Human-Centered Artificial Intelligence User Interfaces to Investigate Fairness](https://mlsanity.com/paper/2206.00474/) (2022)
- [A Human-Centric Take on Model Monitoring](https://mlsanity.com/paper/2206.02868/) (2022)
- [Embracing Contradiction: Theoretical Inconsistency Will Not Impede the Road of Building Responsible AI Systems](https://mlsanity.com/paper/2505.18139/) (2025)
- [Perspectives on Incorporating Expert Feedback into Model Updates](https://mlsanity.com/paper/2205.06905/) (2022)
- [Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools](https://mlsanity.com/paper/2309.17337/) (2023)
- [A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI](https://mlsanity.com/paper/2302.05284/) (2023)
- [Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge](https://mlsanity.com/paper/2308.15700/) (2023)
- [Human-Guided Fair Classification for Natural Language Processing](https://mlsanity.com/paper/2212.10154/) (2022)
- [EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among Stakeholders](https://mlsanity.com/paper/2407.11442/) (2024)
- [FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines](https://mlsanity.com/paper/2307.15475/) (2023)
- [Human-in-the-loop Fairness: Integrating Stakeholder Feedback to Incorporate Fairness Perspectives in Responsible AI](https://mlsanity.com/paper/2312.08064/) (2023)
- [Critical Appraisal of Fairness Metrics in Clinical Predictive AI](https://mlsanity.com/paper/2506.17035/) (2025)
- [Towards Multi-Stakeholder Evaluation of ML Models: A Crowdsourcing Study on Metric Preferences in Job-matching System](https://mlsanity.com/paper/2503.05796/) (2025)
- [Recentering Validity Considerations through Early-Stage Deliberations Around AI and Policy Design](https://mlsanity.com/paper/2303.14602/) (2023)
- [Public Perceptions of Fairness Metrics Across Borders](https://mlsanity.com/paper/2403.16101/) (2024)
- [Revisiting Technical Bias Mitigation Strategies](https://mlsanity.com/paper/2410.17433/) (2024)
- [Laypeople's Attitudes Towards Fair, Affirmative, and Discriminatory Decision-Making Algorithms](https://mlsanity.com/paper/2505.07339/) (2025)
- [RelAItionship Building: Analyzing Recruitment Strategies for Participatory AI](https://mlsanity.com/paper/2508.20176/) (2025)
- ["I think this is fair": Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness Assessment](https://mlsanity.com/paper/2509.17956/) (2025)
- [Empowering Affected Individuals to Shape AI Fairness Assessments: Processes, Criteria, and Tools](https://mlsanity.com/paper/2602.06984/) (2026)
- [Who Should Be Generated? Justifying Demographic Targets in Open-Ended Generation](https://mlsanity.com/paper/2608.02551/) (2026)

## Abstract (arXiv)

Recent work in fair machine learning has proposed dozens of technical definitions of algorithmic fairness and methods for enforcing these definitions. However, we still lack an understanding of how to develop machine learning systems with fairness criteria that reflect relevant stakeholders' nuanced viewpoints in real-world contexts. To address this gap, we propose a framework for eliciting stakeholders' subjective fairness notions. Combining a user interface that allows stakeholders to examine the data and the algorithm's predictions with an interview protocol to probe stakeholders' thoughts while they are interacting with the interface, we can identify stakeholders' fairness beliefs and principles. We conduct a user study to evaluate our framework in the setting of a child maltreatment predictive system. Our evaluations show that the framework allows stakeholders to comprehensively convey their fairness viewpoints. We also discuss how our results can inform the design of predictive systems.