# Improving Identity-Robustness for Face Models

arXiv: https://arxiv.org/abs/2304.03838 · HTML: https://mlsanity.com/paper/2304.03838/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 7 Apr 2023.

## Builds on (most cited first, 24 of 24)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [Adversarial Removal of Demographic Attributes from Text Data](https://mlsanity.com/paper/1808.06640/) (2018)
- [Interpretations are useful: penalizing explanations to align neural networks with prior knowledge](https://mlsanity.com/paper/1909.13584/) (2019)
- [Fairness in Recommendation Ranking through Pairwise Comparisons](https://mlsanity.com/paper/1903.00780/) (2019)
- [Detailed Human Avatars from Monocular Video](https://mlsanity.com/paper/1808.01338/) (2018)
- [Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction](https://mlsanity.com/paper/1807.00199/) (2018)
- [Fair Contrastive Learning for Facial Attribute Classification](https://mlsanity.com/paper/2203.16209/) (2022)
- [Facial Landmark Detection: a Literature Survey](https://mlsanity.com/paper/1805.05563/) (2018)
- [Fairness via Representation Neutralization](https://mlsanity.com/paper/2106.12674/) (2021)
- [img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation](https://mlsanity.com/paper/2012.07791/) (2020)
- [On Tilted Losses in Machine Learning: Theory and Applications](https://mlsanity.com/paper/2109.06141/) (2021)
- [Oversampling for Imbalanced Learning Based on K-Means and SMOTE](https://mlsanity.com/paper/1711.00837/) (2017)
- [Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classification](https://mlsanity.com/paper/2011.06485/) (2020)
- [Learning Social Relation Traits from Face Images](https://mlsanity.com/paper/1509.03936/) (2015)
- [Estimating Structural Disparities for Face Models](https://mlsanity.com/paper/2204.06562/) (2022)

## Built on by (most cited first, 2 of 2)

- [Provable Optimization for Adversarial Fair Self-supervised Contrastive Learning](https://mlsanity.com/paper/2406.05686/) (2024)
- [LLM2Loss: Leveraging Language Models for Explainable Model Diagnostics](https://mlsanity.com/paper/2305.03212/) (2023)

## Abstract (arXiv)

Despite the success of deep-learning models in many tasks, there have been concerns about such models learning shortcuts, and their lack of robustness to irrelevant confounders. When it comes to models directly trained on human faces, a sensitive confounder is that of human identities. Many face-related tasks should ideally be identity-independent, and perform uniformly across different individuals (i.e. be fair). One way to measure and enforce such robustness and performance uniformity is through enforcing it during training, assuming identity-related information is available at scale. However, due to privacy concerns and also the cost of collecting such information, this is often not the case, and most face datasets simply contain input images and their corresponding task-related labels. Thus, improving identity-related robustness without the need for such annotations is of great importance. Here, we explore using face-recognition embedding vectors, as proxies for identities, to enforce such robustness. We propose to use the structure in the face-recognition embedding space, to implicitly emphasize rare samples within each class. We do so by weighting samples according to their conditional inverse density (CID) in the proxy embedding space. Our experiments suggest that such a simple sample weighting scheme, not only improves the training robustness, it often improves the overall performance as a result of such robustness. We also show that employing such constraints during training results in models that are significantly less sensitive to different levels of bias in the dataset.