# Gender Stereotyping Impact in Facial Expression Recognition

arXiv: https://arxiv.org/abs/2210.05332 · HTML: https://mlsanity.com/paper/2210.05332/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 16 of 16)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay](https://mlsanity.com/paper/1803.09820/) (2018)
- [A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle](https://mlsanity.com/paper/1901.10002/) (2019)
- [Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution](https://mlsanity.com/paper/1608.01041/) (2016)
- [Investigating Bias and Fairness in Facial Expression Recognition](https://mlsanity.com/paper/2007.10075/) (2020)
- [Fairness in representation: quantifying stereotyping as a representational harm](https://mlsanity.com/paper/1901.09565/) (2019)
- [Responsible AI: Gender bias assessment in emotion recognition](https://mlsanity.com/paper/2103.11436/) (2021)
- [Uncovering the Bias in Facial Expressions](https://mlsanity.com/paper/2011.11311/) (2020)
- [Bias in Data-driven AI Systems -- An Introductory Survey](https://mlsanity.com/paper/2001.09762/) (2020)
- [Assessing Demographic Bias Transfer from Dataset to Model: A Case Study in Facial Expression Recognition](https://mlsanity.com/paper/2205.10049/) (2022)

## Built on by (most cited first, 8 of 8)

- [Metrics for Dataset Demographic Bias: A Case Study on Facial Expression Recognition](https://mlsanity.com/paper/2303.15889/) (2023)
- [Faces of Fairness: Examining Bias in Facial Expression Recognition Datasets and Models](https://mlsanity.com/paper/2502.11049/) (2025)
- [DSAP: Analyzing Bias Through Demographic Comparison of Datasets](https://mlsanity.com/paper/2312.14626/) (2023)
- [Less can be more: representational vs. stereotypical gender bias in facial expression recognition](https://mlsanity.com/paper/2406.17405/) (2024)
- [Toward Fair Facial Expression Recognition with Improved Distribution Alignment](https://mlsanity.com/paper/2306.06696/) (2023)
- [A Comparison of Human and Machine Learning Errors in Face Recognition](https://mlsanity.com/paper/2502.11337/) (2025)
- [Biased Heritage: How Datasets Shape Models in Facial Expression Recognition](https://mlsanity.com/paper/2503.03446/) (2025)
- [A Feature-level Bias Evaluation Framework for Facial Expression Recognition Models](https://mlsanity.com/paper/2505.20512/) (2025)

## Abstract (arXiv)

Facial Expression Recognition (FER) uses images of faces to identify the emotional state of users, allowing for a closer interaction between humans and autonomous systems. Unfortunately, as the images naturally integrate some demographic information, such as apparent age, gender, and race of the subject, these systems are prone to demographic bias issues. In recent years, machine learning-based models have become the most popular approach to FER. These models require training on large datasets of facial expression images, and their generalization capabilities are strongly related to the characteristics of the dataset. In publicly available FER datasets, apparent gender representation is usually mostly balanced, but their representation in the individual label is not, embedding social stereotypes into the datasets and generating a potential for harm. Although this type of bias has been overlooked so far, it is important to understand the impact it may have in the context of FER. To do so, we use a popular FER dataset, FER+, to generate derivative datasets with different amounts of stereotypical bias by altering the gender proportions of certain labels. We then proceed to measure the discrepancy between the performance of the models trained on these datasets for the apparent gender groups. We observe a discrepancy in the recognition of certain emotions between genders of up to 29 % under the worst bias conditions. Our results also suggest a safety range for stereotypical bias in a dataset that does not appear to produce stereotypical bias in the resulting model. Our findings support the need for a thorough bias analysis of public datasets in problems like FER, where a global balance of demographic representation can still hide other types of bias that harm certain demographic groups.