# Fair Multi-Exit Framework for Facial Attribute Classification

arXiv: https://arxiv.org/abs/2301.02989 · HTML: https://mlsanity.com/paper/2301.02989/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 8 Jan 2023.

## Builds on (most cited first, 16 of 16)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [BERT Loses Patience: Fast and Robust Inference with Early Exit](https://mlsanity.com/paper/2006.04152/) (2020)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Gender Bias in Neural Natural Language Processing](https://mlsanity.com/paper/1807.11714/) (2018)
- [Flexibly Fair Representation Learning by Disentanglement](https://mlsanity.com/paper/1906.02589/) (2019)
- [The Right Tool for the Job: Matching Model and Instance Complexities](https://mlsanity.com/paper/2004.07453/) (2020)
- [Fair Contrastive Learning for Facial Attribute Classification](https://mlsanity.com/paper/2203.16209/) (2022)
- [Fair Feature Distillation for Visual Recognition](https://mlsanity.com/paper/2106.04411/) (2021)
- [Fairness-aware Adversarial Perturbation Towards Bias Mitigation for Deployed Deep Models](https://mlsanity.com/paper/2203.01584/) (2022)
- [Bias Remediation in Driver Drowsiness Detection systems using Generative Adversarial Networks](https://mlsanity.com/paper/1912.12123/) (2019)

## Built on by (most cited first, 4 of 4)

- [Achieve Fairness without Demographics for Dermatological Disease Diagnosis](https://mlsanity.com/paper/2401.08066/) (2024)
- [Toward Fairness via Maximum Mean Discrepancy Regularization on Logits Space](https://mlsanity.com/paper/2402.13061/) (2024)
- [ERDE: Entropy-Regularized Distillation for Early-exit](https://mlsanity.com/paper/2510.04856/) (2025)
- [ProtoFair: Fair Self-Supervised Contrastive Learning via Pseudo-Counterfactual Pairs](https://mlsanity.com/paper/2605.01971/) (2026)

## Abstract (arXiv)

Fairness has become increasingly pivotal in facial recognition. Without bias mitigation, deploying unfair AI would harm the interest of the underprivileged population. In this paper, we observe that though the higher accuracy that features from the deeper layer of a neural networks generally offer, fairness conditions deteriorate as we extract features from deeper layers. This phenomenon motivates us to extend the concept of multi-exit framework. Unlike existing works mainly focusing on accuracy, our multi-exit framework is fairness-oriented, where the internal classifiers are trained to be more accurate and fairer. During inference, any instance with high confidence from an internal classifier is allowed to exit early. Moreover, our framework can be applied to most existing fairness-aware frameworks. Experiment results show that the proposed framework can largely improve the fairness condition over the state-of-the-art in CelebA and UTK Face datasets.