# Testing Group Fairness via Optimal Transport Projections

arXiv: https://arxiv.org/abs/2106.01070 · HTML: https://mlsanity.com/paper/2106.01070/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 43 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 18 of 18)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Empirical Risk Minimization under Fairness Constraints](https://mlsanity.com/paper/1802.08626/) (2018)
- [Counterfactual Fairness in Text Classification through Robustness](https://mlsanity.com/paper/1809.10610/) (2018)
- [Aequitas: A Bias and Fairness Audit Toolkit](https://mlsanity.com/paper/1811.05577/) (2018)
- [Algorithmic Fairness](https://mlsanity.com/paper/2001.09784/) (2020)
- [FlipTest: Fairness Testing via Optimal Transport](https://mlsanity.com/paper/1906.09218/) (2019)
- [Verifying Individual Fairness in Machine Learning Models](https://mlsanity.com/paper/2006.11737/) (2020)
- [A Statistical Test for Probabilistic Fairness](https://mlsanity.com/paper/2012.04800/) (2020)
- [Auditing ML Models for Individual Bias and Unfairness](https://mlsanity.com/paper/2003.05048/) (2020)
- [Confidence Intervals for Testing Disparate Impact in Fair Learning](https://mlsanity.com/paper/1807.06362/) (2018)
- [Distributionally Robust Formulation and Model Selection for the Graphical Lasso](https://mlsanity.com/paper/1905.08975/) (2019)
- [FairTest: Discovering Unwarranted Associations in Data-Driven Applications](https://mlsanity.com/paper/1510.02377/) (2015)

## Built on by (most cited first, 23 of 23)

- [Recent Advances in Optimal Transport for Machine Learning](https://mlsanity.com/paper/2306.16156/) (2023)
- [Statistical Analysis of Wasserstein Distributionally Robust Estimators](https://mlsanity.com/paper/2108.02120/) (2021)
- [Auditing Fairness by Betting](https://mlsanity.com/paper/2305.17570/) (2023)
- [Statistical Inference for Fairness Auditing](https://mlsanity.com/paper/2305.03712/) (2023)
- [Sobolev Transport: A Scalable Metric for Probability Measures with Graph Metrics](https://mlsanity.com/paper/2202.10723/) (2022)
- [Optimal Transport of Classifiers to Fairness](https://mlsanity.com/paper/2202.03814/) (2022)
- [PROSAC: Provably Safe Certification for Machine Learning Models under Adversarial Attacks](https://mlsanity.com/paper/2402.02629/) (2024)
- [Scalable Unbalanced Sobolev Transport for Measures on a Graph](https://mlsanity.com/paper/2302.12498/) (2023)
- [Distributionally Robust Fair Principal Components via Geodesic Descents](https://mlsanity.com/paper/2202.03071/) (2022)
- [OTClean: Data Cleaning for Conditional Independence Violations using Optimal Transport](https://mlsanity.com/paper/2403.02372/) (2024)
- [Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools](https://mlsanity.com/paper/2309.17337/) (2023)
- [Optimal Transport for Measures with Noisy Tree Metric](https://mlsanity.com/paper/2310.13653/) (2023)
- [Robust ML Auditing using Prior Knowledge](https://mlsanity.com/paper/2505.04796/) (2025)
- [Group-blind optimal transport to group parity and its constrained variants](https://mlsanity.com/paper/2310.11407/) (2023)
- [Mitigating Source Bias for Fairer Weak Supervision](https://mlsanity.com/paper/2303.17713/) (2023)
- [FairRec: Fairness Testing for Deep Recommender Systems](https://mlsanity.com/paper/2304.07030/) (2023)
- [Stability Evaluation via Distributional Perturbation Analysis](https://mlsanity.com/paper/2405.03198/) (2024)
- [Auditing and Enforcing Conditional Fairness via Optimal Transport](https://mlsanity.com/paper/2410.14029/) (2024)
- [RobustFair: Adversarial Evaluation through Fairness Confusion Directed Gradient Search](https://mlsanity.com/paper/2305.10906/) (2023)
- [Peer-induced Fairness: A Causal Approach for Algorithmic Fairness Auditing](https://mlsanity.com/paper/2408.02558/) (2024)
- [From Fragile to Certified: Wasserstein Audits of Group Fairness Under Distribution Shift](https://mlsanity.com/paper/2509.26241/) (2025)
- [Cost Efficient Fairness Audit Under Partial Feedback](https://mlsanity.com/paper/2510.03734/) (2025)
- [Wasserstein projection distance for fairness testing of regression models](https://mlsanity.com/paper/2510.04114/) (2025)

## Abstract (arXiv)

We present a statistical testing framework to detect if a given machine learning classifier fails to satisfy a wide range of group fairness notions. The proposed test is a flexible, interpretable, and statistically rigorous tool for auditing whether exhibited biases are intrinsic to the algorithm or due to the randomness in the data. The statistical challenges, which may arise from multiple impact criteria that define group fairness and which are discontinuous on model parameters, are conveniently tackled by projecting the empirical measure onto the set of group-fair probability models using optimal transport. This statistic is efficiently computed using linear programming and its asymptotic distribution is explicitly obtained. The proposed framework can also be used to test for testing composite fairness hypotheses and fairness with multiple sensitive attributes. The optimal transport testing formulation improves interpretability by characterizing the minimal covariate perturbations that eliminate the bias observed in the audit.