# FineFACE: Fair Facial Attribute Classification Leveraging Fine-grained Features

arXiv: https://arxiv.org/abs/2408.16881 · HTML: https://mlsanity.com/paper/2408.16881/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 29 Aug 2024.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Fair Attribute Classification through Latent Space De-biasing](https://mlsanity.com/paper/2012.01469/) (2020)
- [Fair Mixup: Fairness via Interpolation](https://mlsanity.com/paper/2103.06503/) (2021)
- [Analysis of Gender Inequality In Face Recognition Accuracy](https://mlsanity.com/paper/2002.00065/) (2020)
- [Channel Interaction Networks for Fine-Grained Image Categorization](https://mlsanity.com/paper/2003.05235/) (2020)
- [FNNC: Achieving Fairness through Neural Networks](https://mlsanity.com/paper/1811.00247/) (2018)
- [Deep CNNs Meet Global Covariance Pooling: Better Representation and Generalization](https://mlsanity.com/paper/1904.06836/) (2019)
- [FairGRAPE: Fairness-aware GRAdient Pruning mEthod for Face Attribute Classification](https://mlsanity.com/paper/2207.10888/) (2022)
- [Understanding Fairness of Gender Classification Algorithms Across Gender-Race Groups](https://mlsanity.com/paper/2009.11491/) (2020)
- [Learning Semantically Enhanced Feature for Fine-Grained Image Classification](https://mlsanity.com/paper/2006.13457/) (2020)
- [Classes Are Not Equal: An Empirical Study on Image Recognition Fairness](https://mlsanity.com/paper/2402.18133/) (2024)
- [Deep Generative Views to Mitigate Gender Classification Bias Across Gender-Race Groups](https://mlsanity.com/paper/2208.08382/) (2022)
- [A Self-Supervised Learning Pipeline for Demographically Fair Facial Attribute Classification](https://mlsanity.com/paper/2407.10104/) (2024)

## Built on by (most cited first, 3 of 3)

- [Evaluating Fairness and Mitigating Bias in Machine Learning: A Novel Technique using Tensor Data and Bayesian Regression](https://mlsanity.com/paper/2506.11627/) (2025)
- [Time Frequency Analysis of EMG Signal for Gesture Recognition using Fine grained Features](https://mlsanity.com/paper/2504.14708/) (2025)
- [Benchmarking Fairness in Spiking Neural Networks: Data Bias, Spurious Features, and Hardware Effects](https://mlsanity.com/paper/2605.27407/) (2026)

## Abstract (arXiv)

Published research highlights the presence of demographic bias in automated facial attribute classification algorithms, particularly impacting women and individuals with darker skin tones. Existing bias mitigation techniques typically require demographic annotations and often obtain a trade-off between fairness and accuracy, i.e., Pareto inefficiency. Facial attributes, whether common ones like gender or others such as "chubby" or "high cheekbones", exhibit high interclass similarity and intraclass variation across demographics leading to unequal accuracy. This requires the use of local and subtle cues using fine-grained analysis for differentiation. This paper proposes a novel approach to fair facial attribute classification by framing it as a fine-grained classification problem. Our approach effectively integrates both low-level local features (like edges and color) and high-level semantic features (like shapes and structures) through cross-layer mutual attention learning. Here, shallow to deep CNN layers function as experts, offering category predictions and attention regions. An exhaustive evaluation on facial attribute annotated datasets demonstrates that our FineFACE model improves accuracy by 1.32% to 1.74% and fairness by 67% to 83.6%, over the SOTA bias mitigation techniques. Importantly, our approach obtains a Pareto-efficient balance between accuracy and fairness between demographic groups. In addition, our approach does not require demographic annotations and is applicable to diverse downstream classification tasks. To facilitate reproducibility, the code and dataset information is available at https://github.com/VCBSL-Fairness/FineFACE.