# Human-in-the-loop Fairness: Integrating Stakeholder Feedback to Incorporate Fairness Perspectives in Responsible AI

arXiv: https://arxiv.org/abs/2312.08064 · HTML: https://mlsanity.com/paper/2312.08064/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 84 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Builds on (most cited first, 30 of 37)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy](https://mlsanity.com/paper/2002.04087/) (2020)
- [Fairness in Machine Learning: Lessons from Political Philosophy](https://mlsanity.com/paper/1712.03586/) (2017)
- [An Intersectional Definition of Fairness](https://mlsanity.com/paper/1807.08362/) (2018)
- [Fairness Testing: Testing Software for Discrimination](https://mlsanity.com/paper/1709.03221/) (2017)
- [Participation is not a Design Fix for Machine Learning](https://mlsanity.com/paper/2007.02423/) (2020)
- [A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices](https://mlsanity.com/paper/1807.00787/) (2018)
- [FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning](https://mlsanity.com/paper/1904.05419/) (2019)
- ['It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions](https://mlsanity.com/paper/1801.10408/) (2018)
- [Fairlearn: Assessing and Improving Fairness of AI Systems](https://mlsanity.com/paper/2303.16626/) (2023)
- [FairSight: Visual Analytics for Fairness in Decision Making](https://mlsanity.com/paper/1908.00176/) (2019)
- [Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning](https://mlsanity.com/paper/1902.04783/) (2019)
- [FlipTest: Fairness Testing via Optimal Transport](https://mlsanity.com/paper/1906.09218/) (2019)
- [How Different Groups Prioritize Ethical Values for Responsible AI](https://mlsanity.com/paper/2205.07722/) (2022)
- [Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions](https://mlsanity.com/paper/1901.10501/) (2019)
- [FAE: A Fairness-Aware Ensemble Framework](https://mlsanity.com/paper/2002.00695/) (2020)
- [Active Fairness in Algorithmic Decision Making](https://mlsanity.com/paper/1810.00031/) (2018)
- [Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems](https://mlsanity.com/paper/2102.01196/) (2021)
- [Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation](https://mlsanity.com/paper/2302.11623/) (2023)
- [Visual Analysis of Discrimination in Machine Learning](https://mlsanity.com/paper/2007.15182/) (2020)

## Built on by (most cited first, 3 of 3)

- ["I think this is fair": Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness Assessment](https://mlsanity.com/paper/2509.17956/) (2025)
- [Empowering Affected Individuals to Shape AI Fairness Assessments: Processes, Criteria, and Tools](https://mlsanity.com/paper/2602.06984/) (2026)
- [Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI](https://mlsanity.com/paper/2607.09324/) (2026)

## Abstract (arXiv)

Fairness is a growing concern for high-risk decision-making using Artificial Intelligence (AI) but ensuring it through purely technical means is challenging: there is no universally accepted fairness measure, fairness is context-dependent, and there might be conflicting perspectives on what is considered fair. Thus, involving stakeholders, often without a background in AI or fairness, is a promising avenue. Research to directly involve stakeholders is in its infancy, and many questions remain on how to support stakeholders to feedback on fairness, and how this feedback can be integrated into AI models. Our work follows an approach where stakeholders can give feedback on specific decision instances and their outcomes with respect to their fairness, and then to retrain an AI model. In order to investigate this approach, we conducted two studies of a complex AI model for credit rating used in loan applications. In study 1, we collected feedback from 58 lay users on loan application decisions, and conducted offline experiments to investigate the effects on accuracy and fairness metrics. In study 2, we deepened this investigation by showing 66 participants the results of their feedback with respect to fairness, and then conducted further offline analyses. Our work contributes two datasets and associated code frameworks to bootstrap further research, highlights the opportunities and challenges of employing lay user feedback for improving AI fairness, and discusses practical implications for developing AI applications that more closely reflect stakeholder views about fairness.