# From Fragile to Certified: Wasserstein Audits of Group Fairness Under Distribution Shift

arXiv: https://arxiv.org/abs/2509.26241 · HTML: https://mlsanity.com/paper/2509.26241/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 55 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 30 of 33)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [Fair Regression with Wasserstein Barycenters](https://mlsanity.com/paper/2006.07286/) (2020)
- [Distributionally Robust Optimization: A Review](https://mlsanity.com/paper/1908.05659/) (2019)
- [Recent Advances in Optimal Transport for Machine Learning](https://mlsanity.com/paper/2306.16156/) (2023)
- [Robust Fairness under Covariate Shift](https://mlsanity.com/paper/2010.05166/) (2020)
- [Sample Selection for Fair and Robust Training](https://mlsanity.com/paper/2110.14222/) (2021)
- [A Distributionally Robust Approach to Fair Classification](https://mlsanity.com/paper/2007.09530/) (2020)
- [Active Sampling for Min-Max Fairness](https://mlsanity.com/paper/2006.06879/) (2020)
- [Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification](https://mlsanity.com/paper/2301.11562/) (2023)
- [Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference](https://mlsanity.com/paper/2010.09851/) (2020)
- [A Statistical Test for Probabilistic Fairness](https://mlsanity.com/paper/2012.04800/) (2020)
- [Testing Group Fairness via Optimal Transport Projections](https://mlsanity.com/paper/2106.01070/) (2021)
- [Re-weighting Based Group Fairness Regularization via Classwise Robust Optimization](https://mlsanity.com/paper/2303.00442/) (2023)
- [Auditing ML Models for Individual Bias and Unfairness](https://mlsanity.com/paper/2003.05048/) (2020)
- [Wasserstein-based fairness interpretability framework for machine learning models](https://mlsanity.com/paper/2011.03156/) (2020)
- [Fairness in Survival Analysis with Distributionally Robust Optimization](https://mlsanity.com/paper/2409.10538/) (2024)
- [Towards Fairness-Aware Adversarial Learning](https://mlsanity.com/paper/2402.17729/) (2024)
- [Wasserstein Robust Classification with Fairness Constraints](https://mlsanity.com/paper/2103.06828/) (2021)
- [Confidence Intervals for Testing Disparate Impact in Fair Learning](https://mlsanity.com/paper/1807.06362/) (2018)
- [Robust Fairness-aware Learning Under Sample Selection Bias](https://mlsanity.com/paper/2105.11570/) (2021)
- [Fair Wasserstein Coresets](https://mlsanity.com/paper/2311.05436/) (2023)
- [FairWASP: Fast and Optimal Fair Wasserstein Pre-processing](https://mlsanity.com/paper/2311.00109/) (2023)
- [Universal generalization guarantees for Wasserstein distributionally robust models](https://mlsanity.com/paper/2402.11981/) (2024)
- [SLIDE: a surrogate fairness constraint to ensure fairness consistency](https://mlsanity.com/paper/2202.03165/) (2022)
- [Wasserstein Projection Tests: Higher-Order Asymptotics and Connections to Empirical Likelihood](https://mlsanity.com/paper/2408.11753/) (2024)
- [Towards Fair Class-wise Robustness: Class Optimal Distribution Adversarial Training](https://mlsanity.com/paper/2501.04527/) (2025)

## Abstract (arXiv)

Group-fairness metrics (e.g., equalized odds) can vary sharply across resamples and are especially brittle under distribution shift, undermining reliable audits. We propose a Wasserstein distributionally robust framework that certifies worst-case group fairness over a ball of plausible test distributions centered at the empirical law. Our formulation unifies common group fairness notions via a generic conditional-probability functional and defines varepsilon-Wasserstein Distributional Fairness (varepsilon-WDF) as the audit target. Leveraging strong duality, we derive tractable reformulations and an efficient estimator (DRUNE) for varepsilon-WDF. We prove feasibility and consistency and establish finite-sample certification guarantees for auditing fairness, along with quantitative bounds under smoothness and margin conditions. Across standard benchmarks and classifiers, varepsilon-WDF delivers stable fairness assessments under distribution shift, providing a principled basis for auditing and certifying group fairness beyond observational data.