# Empowering Affected Individuals to Shape AI Fairness Assessments: Processes, Criteria, and Tools

arXiv: https://arxiv.org/abs/2602.06984 · HTML: https://mlsanity.com/paper/2602.06984/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 65 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Builds on (most cited first, 25 of 25)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [A survey on datasets for fairness-aware machine learning](https://mlsanity.com/paper/2110.00530/) (2021)
- [Aequitas: A Bias and Fairness Audit Toolkit](https://mlsanity.com/paper/1811.05577/) (2018)
- [FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning](https://mlsanity.com/paper/1904.05419/) (2019)
- [An Empirical Study of Rich Subgroup Fairness for Machine Learning](https://mlsanity.com/paper/1808.08166/) (2018)
- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning](https://mlsanity.com/paper/1902.04783/) (2019)
- [How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairness](https://mlsanity.com/paper/1811.03654/) (2018)
- [Factors Influencing Perceived Fairness in Algorithmic Decision-Making: Algorithm Outcomes, Development Procedures, and Individual Differences](https://mlsanity.com/paper/2001.09604/) (2020)
- [Fairness in Risk Assessment Instruments: Post-Processing to Achieve Counterfactual Equalized Odds](https://mlsanity.com/paper/2009.02841/) (2020)
- [Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems](https://mlsanity.com/paper/2102.01196/) (2021)
- [Making Fair ML Software using Trustworthy Explanation](https://mlsanity.com/paper/2007.02893/) (2020)
- [Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation](https://mlsanity.com/paper/2302.11623/) (2023)
- [EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations](https://mlsanity.com/paper/2402.00491/) (2024)
- [Designing Fair AI for Managing Employees in Organizations: A Review, Critique, and Design Agenda](https://mlsanity.com/paper/2002.09054/) (2020)
- [Fairness-aware Model-agnostic Positive and Unlabeled Learning](https://mlsanity.com/paper/2206.09346/) (2022)
- [Towards Responsible AI: A Design Space Exploration of Human-Centered Artificial Intelligence User Interfaces to Investigate Fairness](https://mlsanity.com/paper/2206.00474/) (2022)
- [Fair Machine Guidance to Enhance Fair Decision Making in Biased People](https://mlsanity.com/paper/2404.05228/) (2024)
- [EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among Stakeholders](https://mlsanity.com/paper/2407.11442/) (2024)
- [Human-in-the-loop Fairness: Integrating Stakeholder Feedback to Incorporate Fairness Perspectives in Responsible AI](https://mlsanity.com/paper/2312.08064/) (2023)
- [A Systematic Literature Review on Equity and Technology in HCI and Fairness: Navigating the Complexities and Nuances of Equity Research](https://mlsanity.com/paper/2501.14886/) (2025)
- [Mindsets and Management: AI and Gender (In)Equitable Access to Finance](https://mlsanity.com/paper/2504.07312/) (2025)

## Abstract (arXiv)

AI systems are increasingly used in high-stakes domains such as credit rating, where fairness concerns are critical. Existing fairness assessments are typically conducted by AI experts or regulators using predefined protected attributes and metrics, which often fail to capture the diversity and nuance of fairness notions held by the individuals who are affected by these systems' decisions, such as decision subjects. Recent work has therefore called for involving affected individuals in fairness assessment, yet little empirical evidence exists on how they create their own fairness criteria or what kinds of criteria they produce - knowledge that could not only inform experts' fairness evaluation and mitigation, but also guide the design of AI assessment tools. We address this gap through a qualitative user study with 18 participants in a credit rating scenario. Participants first articulated their fairness notions in their own words. Then, participants turned them into concrete quantified and operationalized fairness criteria, through an interactive prototype we designed. Our findings provide empirical evidence of the process through which people's fairness notions emerge via grounding in model features, and uncover a diverse set of individuals' custom-defined criteria for both outcome and procedural fairness. We provide design implications for processes and tools that support more inclusive and value-sensitive AI fairness assessment.