# Towards Involving End-users in Interactive Human-in-the-loop AI Fairness

arXiv: https://arxiv.org/abs/2204.10464 · HTML: https://mlsanity.com/paper/2204.10464/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 88 references that are in the map.
- First posted on arXiv 22 Apr 2022.

## Builds on (most cited first, 21 of 21)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Metrics for Explainable AI: Challenges and Prospects](https://mlsanity.com/paper/1812.04608/) (2018)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [Fairness Testing: Testing Software for Discrimination](https://mlsanity.com/paper/1709.03221/) (2017)
- [On the Apparent Conflict Between Individual and Group Fairness](https://mlsanity.com/paper/1912.06883/) (2019)
- [Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making](https://mlsanity.com/paper/1902.02960/) (2019)
- [FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning](https://mlsanity.com/paper/1904.05419/) (2019)
- ['It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions](https://mlsanity.com/paper/1801.10408/) (2018)
- [Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI](https://mlsanity.com/paper/2005.05906/) (2020)
- [Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making](https://mlsanity.com/paper/1802.01029/) (2018)
- [FairSight: Visual Analytics for Fairness in Decision Making](https://mlsanity.com/paper/1908.00176/) (2019)
- [Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning](https://mlsanity.com/paper/1902.04783/) (2019)
- [How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairness](https://mlsanity.com/paper/1811.03654/) (2018)
- [Factors Influencing Perceived Fairness in Algorithmic Decision-Making: Algorithm Outcomes, Development Procedures, and Individual Differences](https://mlsanity.com/paper/2001.09604/) (2020)
- [Do I Look Like a Criminal? Examining how Race Presentation Impacts Human Judgement of Recidivism](https://mlsanity.com/paper/2002.01111/) (2020)
- [Joint Optimization of AI Fairness and Utility: A Human-Centered Approach](https://mlsanity.com/paper/2002.01621/) (2020)

## Built on by (most cited first, 12 of 12)

- [A Critical Survey on Fairness Benefits of Explainable AI](https://mlsanity.com/paper/2310.13007/) (2023)
- [Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness](https://mlsanity.com/paper/2303.00673/) (2023)
- [Human-in-the-loop Fairness: Integrating Stakeholder Feedback to Incorporate Fairness Perspectives in Responsible AI](https://mlsanity.com/paper/2312.08064/) (2023)
- [Explainable AI is Responsible AI: How Explainability Creates Trustworthy and Socially Responsible Artificial Intelligence](https://mlsanity.com/paper/2312.01555/) (2023)
- [Break Out of a Pigeonhole: A Unified Framework for Examining Miscalibration, Bias, and Stereotype in Recommender Systems](https://mlsanity.com/paper/2312.17443/) (2023)
- [Improving Fairness in Adaptive Social Exergames via Shapley Bandits](https://mlsanity.com/paper/2302.09298/) (2023)
- [Interactive Counterfactual Exploration of Algorithmic Harms in Recommender Systems](https://mlsanity.com/paper/2409.06916/) (2024)
- [Understanding Decision Subjects' Engagement with and Perceived Fairness of AI Models When Opportunities of Qualification Improvement Exist](https://mlsanity.com/paper/2410.03126/) (2024)
- [Beyond XAI:Obstacles Towards Responsible AI](https://mlsanity.com/paper/2309.03638/) (2023)
- [Design Requirements for Human-Centered Graph Neural Network Explanations](https://mlsanity.com/paper/2405.06917/) (2024)
- [Noise-Free Explanation for Driving Action Prediction](https://mlsanity.com/paper/2407.06339/) (2024)
- [TA3: Testing Against Adversarial Attacks on Machine Learning Models](https://mlsanity.com/paper/2410.05334/) (2024)

## Abstract (arXiv)

Ensuring fairness in artificial intelligence (AI) is important to counteract bias and discrimination in far-reaching applications. Recent work has started to investigate how humans judge fairness and how to support machine learning (ML) experts in making their AI models fairer. Drawing inspiration from an Explainable AI (XAI) approach called explanatory debugging used in interactive machine learning, our work explores designing interpretable and interactive human-in-the-loop interfaces that allow ordinary end-users without any technical or domain background to identify potential fairness issues and possibly fix them in the context of loan decisions. Through workshops with end-users, we co-designed and implemented a prototype system that allowed end-users to see why predictions were made, and then to change weights on features to "debug" fairness issues. We evaluated the use of this prototype system through an online study. To investigate the implications of diverse human values about fairness around the globe, we also explored how cultural dimensions might play a role in using this prototype. Our results contribute to the design of interfaces to allow end-users to be involved in judging and addressing AI fairness through a human-in-the-loop approach.