# Estimating Structural Disparities for Face Models

arXiv: https://arxiv.org/abs/2204.06562 · HTML: https://mlsanity.com/paper/2204.06562/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 13 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [Environment Inference for Invariant Learning](https://mlsanity.com/paper/2010.07249/) (2020)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace](https://mlsanity.com/paper/1910.04855/) (2019)
- [Fairness in Machine Learning](https://mlsanity.com/paper/2012.15816/) (2020)
- [FairGAN: Fairness-aware Generative Adversarial Networks](https://mlsanity.com/paper/1805.11202/) (2018)
- [Ethical Machine Learning in Health Care](https://mlsanity.com/paper/2009.10576/) (2020)
- [Context-Aware Emotion Recognition Networks](https://mlsanity.com/paper/1908.05913/) (2019)
- [Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved](https://mlsanity.com/paper/1811.11154/) (2018)
- [Shake-Shake regularization](https://mlsanity.com/paper/1705.07485/) (2017)
- [Investigating Bias and Fairness in Facial Expression Recognition](https://mlsanity.com/paper/2007.10075/) (2020)
- [Context Based Emotion Recognition using EMOTIC Dataset](https://mlsanity.com/paper/2003.13401/) (2020)
- [M3T: Multi-Modal Continuous Valence-Arousal Estimation in the Wild](https://mlsanity.com/paper/2002.02957/) (2020)
- [Deep Convolutional Neural Network for Age Estimation based on VGG-Face Model](https://mlsanity.com/paper/1709.01664/) (2017)

## Built on by (most cited first, 4 of 4)

- [Zero-shot Visual Commonsense Immorality Prediction](https://mlsanity.com/paper/2211.05521/) (2022)
- [Improving Identity-Robustness for Face Models](https://mlsanity.com/paper/2304.03838/) (2023)
- [LLM2Loss: Leveraging Language Models for Explainable Model Diagnostics](https://mlsanity.com/paper/2305.03212/) (2023)
- [Bias Mitigation in Face Recognition via Demographic-based Supervised Contrastive Learning](https://mlsanity.com/paper/2608.12971/) (2026)

## Abstract (arXiv)

In machine learning, disparity metrics are often defined by measuring the difference in the performance or outcome of a model, across different sub-populations (groups) of datapoints. Thus, the inputs to disparity quantification consist of a model's predictions haty, the ground-truth labels for the predictions y, and group labels g for the data points. Performance of the model for each group is calculated by comparing haty and y for the datapoints within a specific group, and as a result, disparity of performance across the different groups can be calculated. In many real world scenarios however, group labels (g) may not be available at scale during training and validation time, or collecting them might not be feasible or desirable as they could often be sensitive information. As a result, evaluating disparity metrics across categorical groups would not be feasible. On the other hand, in many scenarios noisy groupings may be obtainable using some form of a proxy, which would allow measuring disparity metrics across sub-populations. Here we explore performing such analysis on computer vision models trained on human faces, and on tasks such as face attribute prediction and affect estimation. Our experiments indicate that embeddings resulting from an off-the-shelf face recognition model, could meaningfully serve as a proxy for such estimation.