# Reliable and Reproducible Demographic Inference for Fairness in Face Analysis

arXiv: https://arxiv.org/abs/2510.20482 · HTML: https://mlsanity.com/paper/2510.20482/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 60 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [Fair Regression: Quantitative Definitions and Reduction-based Algorithms](https://mlsanity.com/paper/1905.12843/) (2019)
- [DCFace: Synthetic Face Generation with Dual Condition Diffusion Model](https://mlsanity.com/paper/2304.07060/) (2023)
- [Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings](https://mlsanity.com/paper/1809.02169/) (2018)
- [A Clarification of the Nuances in the Fairness Metrics Landscape](https://mlsanity.com/paper/2106.00467/) (2021)
- [Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing](https://mlsanity.com/paper/2001.00964/) (2020)
- [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021)
- [Face Recognition: Too Bias, or Not Too Bias?](https://mlsanity.com/paper/2002.06483/) (2020)
- [Diversity in Faces](https://mlsanity.com/paper/1901.10436/) (2019)
- [Mitigating Face Recognition Bias via Group Adaptive Classifier](https://mlsanity.com/paper/2006.07576/) (2020)
- [OpenBias: Open-set Bias Detection in Text-to-Image Generative Models](https://mlsanity.com/paper/2404.07990/) (2024)
- [Meta Balanced Network for Fair Face Recognition](https://mlsanity.com/paper/2205.06548/) (2022)
- [VHELM: A Holistic Evaluation of Vision Language Models](https://mlsanity.com/paper/2410.07112/) (2024)
- [SensitiveLoss: Improving Accuracy and Fairness of Face Representations with Discrimination-Aware Deep Learning](https://mlsanity.com/paper/2004.11246/) (2020)
- [Analysis of Gender Inequality In Face Recognition Accuracy](https://mlsanity.com/paper/2002.00065/) (2020)
- [WebFace260M: A Benchmark for Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2204.10149/) (2022)
- [How Good is ChatGPT at Face Biometrics? A First Look into Recognition, Soft Biometrics, and Explainability](https://mlsanity.com/paper/2401.13641/) (2024)
- [One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision](https://mlsanity.com/paper/2102.02320/) (2021)
- [AI-Face: A Million-Scale Demographically Annotated AI-Generated Face Dataset and Fairness Benchmark](https://mlsanity.com/paper/2406.00783/) (2024)
- [About Face: A Survey of Facial Recognition Evaluation](https://mlsanity.com/paper/2102.00813/) (2021)
- [Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition](https://mlsanity.com/paper/2210.09943/) (2022)
- [Deep Learning for Face Recognition: Pride or Prejudiced?](https://mlsanity.com/paper/1904.01219/) (2019)
- [FaceXBench: Evaluating Multimodal LLMs on Face Understanding](https://mlsanity.com/paper/2501.10360/) (2025)
- [Rethinking Common Assumptions to Mitigate Racial Bias in Face Recognition Datasets](https://mlsanity.com/paper/2109.03229/) (2021)
- [Balancing Biases and Preserving Privacy on Balanced Faces in the Wild](https://mlsanity.com/paper/2103.09118/) (2021)

## Abstract (arXiv)

Fairness evaluation in face analysis systems (FAS) typically depends on automatic demographic attribute inference (DAI), which itself relies on predefined demographic segmentation. However, the validity of fairness auditing hinges on the reliability of the DAI process. We begin by providing a theoretical motivation for this dependency, showing that improved DAI reliability leads to less biased and lower-variance estimates of FAS fairness. To address this, we propose a fully reproducible DAI pipeline that replaces conventional end-to-end training with a modular transfer learning approach. Our design integrates pretrained face recognition encoders with non-linear classification heads. We audit this pipeline across three dimensions: accuracy, fairness, and a newly introduced notion of robustness, defined via intra-identity consistency. The proposed robustness metric is applicable to any demographic segmentation scheme. We benchmark the pipeline on gender and ethnicity inference across multiple datasets and training setups. Our results show that the proposed method outperforms strong baselines, particularly on ethnicity, which is the more challenging attribute. To promote transparency and reproducibility, we will publicly release the training dataset metadata, full codebase, pretrained models, and evaluation toolkit. This work contributes a reliable foundation for demographic inference in fairness auditing.