# Discovering Intersectional Bias via Directional Alignment in Face Recognition Embeddings

arXiv: https://arxiv.org/abs/2510.15520 · HTML: https://mlsanity.com/paper/2510.15520/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 54 references that are in the map.
- First posted on arXiv 17 Oct 2025.

## Builds on (most cited first, 30 of 36)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems](https://mlsanity.com/paper/2011.12945/) (2020)
- [Ovis: Structural Embedding Alignment for Multimodal Large Language Model](https://mlsanity.com/paper/2405.20797/) (2024)
- [Partial FC: Training 10 Million Identities on a Single Machine](https://mlsanity.com/paper/2010.05222/) (2020)
- [Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learning](https://mlsanity.com/paper/1911.10692/) (2019)
- [ElasticFace: Elastic Margin Loss for Deep Face Recognition](https://mlsanity.com/paper/2109.09416/) (2021)
- [Arc2Face: A Foundation Model for ID-Consistent Human Faces](https://mlsanity.com/paper/2403.11641/) (2024)
- [Demographic Bias in Biometrics: A Survey on an Emerging Challenge](https://mlsanity.com/paper/2003.02488/) (2020)
- [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021)
- [Ola: Pushing the Frontiers of Omni-Modal Language Model](https://mlsanity.com/paper/2502.04328/) (2025)
- [Face Recognition: Too Bias, or Not Too Bias?](https://mlsanity.com/paper/2002.06483/) (2020)
- [Mitigating Face Recognition Bias via Group Adaptive Classifier](https://mlsanity.com/paper/2006.07576/) (2020)
- [Towards causal benchmarking of bias in face analysis algorithms](https://mlsanity.com/paper/2007.06570/) (2020)
- [Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?](https://mlsanity.com/paper/1912.07398/) (2019)
- [Scalable Vision Language Model Training via High Quality Data Curation](https://mlsanity.com/paper/2501.05952/) (2025)
- [LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions](https://mlsanity.com/paper/2104.00820/) (2021)
- [SensitiveLoss: Improving Accuracy and Fairness of Face Representations with Discrimination-Aware Deep Learning](https://mlsanity.com/paper/2004.11246/) (2020)
- [Gender Slopes: Counterfactual Fairness for Computer Vision Models by Attribute Manipulation](https://mlsanity.com/paper/2005.10430/) (2020)
- [InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity](https://mlsanity.com/paper/2503.16418/) (2025)
- [Disentangling Neuron Representations with Concept Vectors](https://mlsanity.com/paper/2304.09707/) (2023)
- [Benchmarking Algorithmic Bias in Face Recognition: An Experimental Approach Using Synthetic Faces and Human Evaluation](https://mlsanity.com/paper/2308.05441/) (2023)

## Built on by (most cited first, 2 of 2)

- [Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision](https://mlsanity.com/paper/2607.07236/) (2026)
- [SCOUT: Semantic Concept Discovery for Open-Vocabulary Editing of face Recognition Templates](https://mlsanity.com/paper/2608.16251/) (2026)

## Abstract (arXiv)

Modern face recognition models embed identities on a unit hypersphere, where identity variation forms tight clusters. Conversely, shared semantic attributes can often be effectively approximated as linear directions in the latent space. Existing bias evaluation methods rely on predefined attribute labels, synthetic counterfactuals, or proximity-based clustering, all of which fail to capture intersectional subpopulations that emerge along latent directions. We introduce LatentAlign, an attribute-free algorithm that discovers semantically coherent and interpretable subpopulations by iteratively aligning embeddings along dominant latent directions. Unlike distance-based clustering, LatentAlign exploits the geometry of hyperspherical embeddings to isolate directional structures shared across identities, allowing for the interpretable discovery of attributes. Across four state-of-the-art recognition backbones (ArcFace, CosFace, ElasticFace, PartialFC) and two benchmarks (RFW, CelebA), LatentAlign consistently yields more semantically coherent groups than k-means, spherical k-means, nearest-neighbor search, and DBSCAN. Crucially, the discovered subpopulations expose severe intersectional vulnerabilities, with False Match Rates up to 4x higher than groups defined by explicit annotations. Our results show that by treating semantic attributes as directional features rather than spatial clusters, we can effectively isolate intersectional subpopulations and expose hidden biases that standard audits miss.