# Oxford Handbook on AI Ethics Book Chapter on Race and Gender

arXiv: https://arxiv.org/abs/1908.06165 · HTML: https://mlsanity.com/paper/1908.06165/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2019.

## Built on by (most cited first, 7 of 7)

- [Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments](https://mlsanity.com/paper/2203.11764/) (2022)
- [Black Feminist Musings on Algorithmic Oppression](https://mlsanity.com/paper/2101.09869/) (2021)
- [Fairness and Bias in Robot Learning](https://mlsanity.com/paper/2207.03444/) (2022)
- [AI and Blackness: Towards moving beyond bias and representation](https://mlsanity.com/paper/2111.03687/) (2021)
- [FATE in AI: Towards Algorithmic Inclusivity and Accessibility](https://mlsanity.com/paper/2301.01590/) (2023)
- [Sociocultural knowledge is needed for selection of shots in hate speech detection tasks](https://mlsanity.com/paper/2304.01890/) (2023)
- [It's complicated. The relationship of algorithmic fairness and non-discrimination provisions for high-risk systems in the EU AI Act](https://mlsanity.com/paper/2501.12962/) (2025)

## Abstract (arXiv)

From massive face-recognition-based surveillance and machine-learning-based decision systems predicting crime recidivism rates, to the move towards automated health diagnostic systems, artificial intelligence (AI) is being used in scenarios that have serious consequences in people's lives. However, this rapid permeation of AI into society has not been accompanied by a thorough investigation of the sociopolitical issues that cause certain groups of people to be harmed rather than advantaged by it. For instance, recent studies have shown that commercial face recognition systems have much higher error rates for dark skinned women while having minimal errors on light skinned men. A 2016 ProPublica investigation uncovered that machine learning based tools that assess crime recidivism rates in the US are biased against African Americans. Other studies show that natural language processing tools trained on newspapers exhibit societal biases (e.g. finishing the analogy "Man is to computer programmer as woman is to X" by homemaker). At the same time, books such as Weapons of Math Destruction and Automated Inequality detail how people in lower socioeconomic classes in the US are subjected to more automated decision making tools than those who are in the upper class. Thus, these tools are most often used on people towards whom they exhibit the most bias. While many technical solutions have been proposed to alleviate bias in machine learning systems, we have to take a holistic and multifaceted approach. This includes standardization bodies determining what types of systems can be used in which scenarios, making sure that automated decision tools are created by people from diverse backgrounds, and understanding the historical and political factors that disadvantage certain groups who are subjected to these tools.