# Learning Fair Face Representation With Progressive Cross Transformer

arXiv: https://arxiv.org/abs/2108.04983 · HTML: https://mlsanity.com/paper/2108.04983/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 57 references that are in the map.
- First posted on arXiv 11 Aug 2021.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Bottleneck Transformers for Visual Recognition](https://mlsanity.com/paper/2101.11605/) (2021)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [Probabilistic Face Embeddings](https://mlsanity.com/paper/1904.09658/) (2019)
- [Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learning](https://mlsanity.com/paper/1911.10692/) (2019)
- [Demographic Bias in Biometrics: A Survey on an Emerging Challenge](https://mlsanity.com/paper/2003.02488/) (2020)
- [Deep Imbalanced Learning for Face Recognition and Attribute Prediction](https://mlsanity.com/paper/1806.00194/) (2018)
- [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021)
- [Face Recognition: Too Bias, or Not Too Bias?](https://mlsanity.com/paper/2002.06483/) (2020)
- [Diversity in Faces](https://mlsanity.com/paper/1901.10436/) (2019)
- [Generative Adversarial Minority Oversampling](https://mlsanity.com/paper/1903.09730/) (2019)
- [Mitigating Face Recognition Bias via Group Adaptive Classifier](https://mlsanity.com/paper/2006.07576/) (2020)
- [Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?](https://mlsanity.com/paper/1912.07398/) (2019)

## Built on by (most cited first, 4 of 4)

- [Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Survey](https://mlsanity.com/paper/2211.05617/) (2022)
- [How to Boost Face Recognition with StyleGAN?](https://mlsanity.com/paper/2210.10090/) (2022)
- [The Impact of Racial Distribution in Training Data on Face Recognition Bias: A Closer Look](https://mlsanity.com/paper/2211.14498/) (2022)
- [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025)

## Abstract (arXiv)

Face recognition (FR) has made extraordinary progress owing to the advancement of deep convolutional neural networks. However, demographic bias among different racial cohorts still challenges the practical face recognition system. The race factor has been proven to be a dilemma for fair FR (FFR) as the subject-related specific attributes induce the classification bias whilst carrying some useful cues for FR. To mitigate racial bias and meantime preserve robust FR, we abstract face identity-related representation as a signal denoising problem and propose a progressive cross transformer (PCT) method for fair face recognition. Originating from the signal decomposition theory, we attempt to decouple face representation into i) identity-related components and ii) noisy/identity-unrelated components induced by race. As an extension of signal subspace decomposition, we formulate face decoupling as a generalized functional expression model to cross-predict face identity and race information. The face expression model is further concretized by designing dual cross-transformers to distill identity-related components and suppress racial noises. In order to refine face representation, we take a progressive face decoupling way to learn identity/race-specific transformations, so that identity-unrelated components induced by race could be better disentangled. We evaluate the proposed PCT on the public fair face recognition benchmarks (BFW, RFW) and verify that PCT is capable of mitigating bias in face recognition while achieving state-of-the-art FR performance. Besides, visualization results also show that the attention maps in PCT can well reveal the race-related/biased facial regions.