# Toward Fair Facial Expression Recognition with Improved Distribution Alignment

arXiv: https://arxiv.org/abs/2306.06696 · HTML: https://mlsanity.com/paper/2306.06696/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 11 Jun 2023.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines](https://mlsanity.com/paper/1810.12488/) (2018)
- [The Variational Fair Autoencoder](https://mlsanity.com/paper/1511.00830/) (2015)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings](https://mlsanity.com/paper/1809.02169/) (2018)
- [Investigating Bias and Fairness in Facial Expression Recognition](https://mlsanity.com/paper/2007.10075/) (2020)
- [Bias Mitigation Post-processing for Individual and Group Fairness](https://mlsanity.com/paper/1812.06135/) (2018)
- [Understanding and Mitigating Annotation Bias in Facial Expression Recognition](https://mlsanity.com/paper/2108.08504/) (2021)
- [Exploring Disentangled Feature Representation Beyond Face Identification](https://mlsanity.com/paper/1804.03487/) (2018)
- [Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition](https://mlsanity.com/paper/2103.08637/) (2021)
- [Kernel Mean Shrinkage Estimators](https://mlsanity.com/paper/1405.5505/) (2014)
- [Bias Remediation in Driver Drowsiness Detection systems using Generative Adversarial Networks](https://mlsanity.com/paper/1912.12123/) (2019)
- [The Effect of Model Compression on Fairness in Facial Expression Recognition](https://mlsanity.com/paper/2201.01709/) (2022)
- [Gender Stereotyping Impact in Facial Expression Recognition](https://mlsanity.com/paper/2210.05332/) (2022)
- [Face Trees for Expression Recognition](https://mlsanity.com/paper/2112.02487/) (2021)
- [Analysis of Semi-Supervised Methods for Facial Expression Recognition](https://mlsanity.com/paper/2208.00544/) (2022)
- [FaceTopoNet: Facial Expression Recognition using Face Topology Learning](https://mlsanity.com/paper/2209.06322/) (2022)

## Built on by (most cited first, 2 of 2)

- [Some Optimizers are More Equal: Understanding the Role of Optimizers in Group Fairness](https://mlsanity.com/paper/2504.14882/) (2025)
- [A Feature-level Bias Evaluation Framework for Facial Expression Recognition Models](https://mlsanity.com/paper/2505.20512/) (2025)

## Abstract (arXiv)

We present a novel approach to mitigate bias in facial expression recognition (FER) models. Our method aims to reduce sensitive attribute information such as gender, age, or race, in the embeddings produced by FER models. We employ a kernel mean shrinkage estimator to estimate the kernel mean of the distributions of the embeddings associated with different sensitive attribute groups, such as young and old, in the Hilbert space. Using this estimation, we calculate the maximum mean discrepancy (MMD) distance between the distributions and incorporate it in the classifier loss along with an adversarial loss, which is then minimized through the learning process to improve the distribution alignment. Our method makes sensitive attributes less recognizable for the model, which in turn promotes fairness. Additionally, for the first time, we analyze the notion of attractiveness as an important sensitive attribute in FER models and demonstrate that FER models can indeed exhibit biases towards more attractive faces. To prove the efficacy of our model in reducing bias regarding different sensitive attributes (including the newly proposed attractiveness attribute), we perform several experiments on two widely used datasets, CelebA and RAF-DB. The results in terms of both accuracy and fairness measures outperform the state-of-the-art in most cases, demonstrating the effectiveness of the proposed method.