# Inclusive Portraits: Race-Aware Human-in-the-Loop Technology

arXiv: https://arxiv.org/abs/2311.03567 · HTML: https://mlsanity.com/paper/2311.03567/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 100 references that are in the map.
- First posted on arXiv 6 Nov 2023.

## Builds on (most cited first, 18 of 18)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learning](https://mlsanity.com/paper/1911.10692/) (2019)
- [Jury Learning: Integrating Dissenting Voices into Machine Learning Models](https://mlsanity.com/paper/2202.02950/) (2022)
- [Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing](https://mlsanity.com/paper/2001.00964/) (2020)
- [Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development](https://mlsanity.com/paper/2108.04308/) (2021)
- [Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?](https://mlsanity.com/paper/1912.07398/) (2019)
- [A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk](https://mlsanity.com/paper/1712.05796/) (2017)
- [FairFace Challenge at ECCV 2020: Analyzing Bias in Face Recognition](https://mlsanity.com/paper/2009.07838/) (2020)
- [Deep learning and face recognition: the state of the art](https://mlsanity.com/paper/1902.03524/) (2019)
- [Designing Closed Human-in-the-loop Deferral Pipelines](https://mlsanity.com/paper/2202.04718/) (2022)
- [Datavoidant: An AI System for Addressing Political Data Voids on Social Media](https://mlsanity.com/paper/2210.13594/) (2022)
- [Recent Advances in Deep Learning Techniques for Face Recognition](https://mlsanity.com/paper/2103.10492/) (2021)
- [Crowd Coach: Peer Coaching for Crowd Workers' Skill Growth](https://mlsanity.com/paper/1811.05364/) (2018)
- [Fighting Disaster Misinformation in Latin America: The #19S Mexican Earthquake Case Study](https://mlsanity.com/paper/2007.05848/) (2020)
- [The Challenges of Crowd Workers in Rural and Urban America](https://mlsanity.com/paper/2012.15211/) (2020)
- [The Global Care Ecosystems of 3D Printed Assistive Devices](https://mlsanity.com/paper/2210.14984/) (2022)

## Built on by (most cited first, 3 of 3)

- [Engaging with Children's Artwork in Mixed Visual-Ability Families](https://mlsanity.com/paper/2407.18874/) (2024)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [A Comparison of Human and Machine Learning Errors in Face Recognition](https://mlsanity.com/paper/2502.11337/) (2025)

## Abstract (arXiv)

AI has revolutionized the processing of various services, including the automatic facial verification of people. Automated approaches have demonstrated their speed and efficiency in verifying a large volume of faces, but they can face challenges when processing content from certain communities, including communities of people of color. This challenge has prompted the adoption of "human-in-the-loop" (HITL) approaches, where human workers collaborate with the AI to minimize errors. However, most HITL approaches do not consider workers' individual characteristics and backgrounds. This paper proposes a new approach, called Inclusive Portraits (IP), that connects with social theories around race to design a racially-aware human-in-the-loop system. Our experiments have provided evidence that incorporating race into human-in-the-loop (HITL) systems for facial verification can significantly enhance performance, especially for services delivered to people of color. Our findings also highlight the importance of considering individual worker characteristics in the design of HITL systems, rather than treating workers as a homogenous group. Our research has significant design implications for developing AI-enhanced services that are more inclusive and equitable.