# Towards Fair Affective Robotics: Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition

arXiv: https://arxiv.org/abs/2103.09233 · HTML: https://mlsanity.com/paper/2103.09233/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 15 Mar 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines](https://mlsanity.com/paper/1810.12488/) (2018)
- [Latent Replay for Real-Time Continual Learning](https://mlsanity.com/paper/1912.01100/) (2019)
- [Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challenges](https://mlsanity.com/paper/1907.00182/) (2019)
- [Investigating Bias and Fairness in Facial Expression Recognition](https://mlsanity.com/paper/2007.10075/) (2020)
- [Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment](https://mlsanity.com/paper/1803.05588/) (2018)
- [Exploring Racial Bias within Face Recognition via per-subject Adversarially-Enabled Data Augmentation](https://mlsanity.com/paper/2004.08945/) (2020)
- [Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization](https://mlsanity.com/paper/1805.10966/) (2018)
- [Exploring Disentangled Feature Representation Beyond Face Identification](https://mlsanity.com/paper/1804.03487/) (2018)
- [A Deeper Look at Facial Expression Dataset Bias](https://mlsanity.com/paper/1904.11150/) (2019)
- [Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives](https://mlsanity.com/paper/1809.08927/) (2018)
- [Creating a Robot Coach for Mindfulness and Wellbeing: A Longitudinal Study](https://mlsanity.com/paper/2006.05289/) (2020)
- [Spatio-Temporal Analysis of Facial Actions using Lifecycle-Aware Capsule Networks](https://mlsanity.com/paper/2011.08819/) (2020)

## Built on by (most cited first, 3 of 3)

- [Bias Reducing Multitask Learning on Mental Health Prediction](https://mlsanity.com/paper/2208.03621/) (2022)
- [Fairness Evolution in Continual Learning for Medical Imaging](https://mlsanity.com/paper/2406.02480/) (2024)
- [One-Frame Calibration with Siamese Network in Facial Action Unit Recognition](https://mlsanity.com/paper/2409.00240/) (2024)

## Abstract (arXiv)

As affective robots become integral in human life, these agents must be able to fairly evaluate human affective expressions without discriminating against specific demographic groups. Identifying bias in Machine Learning (ML) systems as a critical problem, different approaches have been proposed to mitigate such biases in the models both at data and algorithmic levels. In this work, we propose Continual Learning (CL) as an effective strategy to enhance fairness in Facial Expression Recognition (FER) systems, guarding against biases arising from imbalances in data distributions. We compare different state-of-the-art bias mitigation approaches with CL-based strategies for fairness on expression recognition and Action Unit (AU) detection tasks using popular benchmarks for each; RAF-DB and BP4D. Our experiments show that CL-based methods, on average, outperform popular bias mitigation techniques, strengthening the need for further investigation into CL for the development of fairer FER algorithms.