# An Experimental Evaluation of Covariates Effects on Unconstrained Face Verification

arXiv: https://arxiv.org/abs/1808.05508 · HTML: https://mlsanity.com/paper/1808.05508/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2018.

## Built on by (most cited first, 28 of 28)

- [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021)
- [PASS: Protected Attribute Suppression System for Mitigating Bias in Face Recognition](https://mlsanity.com/paper/2108.03764/) (2021)
- [Gendered Differences in Face Recognition Accuracy Explained by Hairstyles, Makeup, and Facial Morphology](https://mlsanity.com/paper/2112.14656/) (2021)
- [AI-Face: A Million-Scale Demographically Annotated AI-Generated Face Dataset and Fairness Benchmark](https://mlsanity.com/paper/2406.00783/) (2024)
- [Face Recognition Accuracy Across Demographics: Shining a Light Into the Problem](https://mlsanity.com/paper/2206.01881/) (2022)
- [Benchmarking Algorithmic Bias in Face Recognition: An Experimental Approach Using Synthetic Faces and Human Evaluation](https://mlsanity.com/paper/2308.05441/) (2023)
- [On Modality Bias Recognition and Reduction](https://mlsanity.com/paper/2202.12690/) (2022)
- [Analysis of Manual and Automated Skin Tone Assignments for Face Recognition Applications](https://mlsanity.com/paper/2104.14685/) (2021)
- [Reliability and Validity of Image-Based and Self-Reported Skin Phenotype Metrics](https://mlsanity.com/paper/2106.11240/) (2021)
- [The Gender Gap in Face Recognition Accuracy Is a Hairy Problem](https://mlsanity.com/paper/2206.04867/) (2022)
- [Unravelling the Effect of Image Distortions for Biased Prediction of Pre-trained Face Recognition Models](https://mlsanity.com/paper/2108.06581/) (2021)
- [Measuring Bias in AI Models: An Statistical Approach Introducing N-Sigma](https://mlsanity.com/paper/2304.13680/) (2023)
- [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025)
- [Distill and De-bias: Mitigating Bias in Face Verification using Knowledge Distillation](https://mlsanity.com/paper/2112.09786/) (2021)
- [Casual Conversations v2: Designing a large consent-driven dataset to measure algorithmic bias and robustness](https://mlsanity.com/paper/2211.05809/) (2022)
- [Beard Segmentation and Recognition Bias](https://mlsanity.com/paper/2308.15740/) (2023)
- [Can the accuracy bias by facial hairstyle be reduced through balancing the training data?](https://mlsanity.com/paper/2405.20062/) (2024)
- [Matched sample selection with GANs for mitigating attribute confounding](https://mlsanity.com/paper/2103.13455/) (2021)
- [Does Face Recognition Error Echo Gender Classification Error?](https://mlsanity.com/paper/2104.13803/) (2021)
- [Learning Fair Face Representation With Progressive Cross Transformer](https://mlsanity.com/paper/2108.04983/) (2021)
- [FaceQgen: Semi-Supervised Deep Learning for Face Image Quality Assessment](https://mlsanity.com/paper/2201.00770/) (2022)
- [Pose Impact Estimation on Face Recognition using 3D-Aware Synthetic Data with Application to Quality Assessment](https://mlsanity.com/paper/2303.00491/) (2023)
- [CounterFace: A Synthetic Face Dataset for Fine-Grained Counterfactual Evaluation of Face Recognition Systems](https://mlsanity.com/paper/2407.13922/) (2024)
- [Sum of Group Error Differences: A Critical Examination of Bias Evaluation in Biometric Verification and a Dual-Metric Measure](https://mlsanity.com/paper/2404.15385/) (2024)
- [SELM: Siamese Extreme Learning Machine with Application to Face Biometrics](https://mlsanity.com/paper/2108.03140/) (2021)
- [Unsupervised Domain-Specific Deblurring using Scale-Specific Attention](https://mlsanity.com/paper/2112.06175/) (2021)
- [Security and Privacy Product Inclusion](https://mlsanity.com/paper/2404.13220/) (2024)
- [Mitigating Bias with Words: Inducing Demographic Ambiguity in Face Recognition Templates by Text Encoding](https://mlsanity.com/paper/2512.08981/) (2025)

## Abstract (arXiv)

Covariates are factors that have a debilitating influence on face verification performance. In this paper, we comprehensively study two covariate related problems for unconstrained face verification: first, how covariates affect the performance of deep neural networks on the large-scale unconstrained face verification problem; second, how to utilize covariates to improve verification performance. To study the first problem, we implement five state-of-the-art deep convolutional networks (DCNNs) for face verification and evaluate them on three challenging covariates datasets. In total, seven covariates are considered: pose (yaw and roll), age, facial hair, gender, indoor/outdoor, occlusion (nose and mouth visibility, eyes visibility, and forehead visibility), and skin tone. These covariates cover both intrinsic subject-specific characteristics and extrinsic factors of faces. Some of the results confirm and extend the findings of previous studies, others are new findings that were rarely mentioned previously or did not show consistent trends. For the second problem, we demonstrate that with the assistance of gender information, the quality of a pre-curated noisy large-scale face dataset for face recognition can be further improved. After retraining the face recognition model using the curated data, performance improvement is observed at low False Acceptance Rates (FARs) (FAR=10-5, 10-6, 10-7).