# Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks

arXiv: https://arxiv.org/abs/2507.20708 · HTML: https://mlsanity.com/paper/2507.20708/

## Key facts

- Cited by 2 papers in the map (data through 7 Oct 2026).
- Builds on 29 of its 60 references that are in the map.
- First posted on arXiv 28 Jul 2025.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Fairness in Machine Learning](https://mlsanity.com/paper/2012.15816/) (2020)
- [The Road Less Scheduled](https://mlsanity.com/paper/2405.15682/) (2024)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [Fair Regression with Wasserstein Barycenters](https://mlsanity.com/paper/2006.07286/) (2020)
- [Fairwashing: the risk of rationalization](https://mlsanity.com/paper/1901.09749/) (2019)
- [Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluation](https://mlsanity.com/paper/2211.13358/) (2022)
- [A survey of bias in Machine Learning through the prism of Statistical Parity for the Adult Data Set](https://mlsanity.com/paper/2003.14263/) (2020)
- [Active Fairness Auditing](https://mlsanity.com/paper/2206.08450/) (2022)
- [Statistical Inference for Fairness Auditing](https://mlsanity.com/paper/2305.03712/) (2023)
- [Auditing Work: Exploring the New York City algorithmic bias audit regime](https://mlsanity.com/paper/2402.08101/) (2024)
- [From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing](https://mlsanity.com/paper/2410.04772/) (2024)
- [Faking Fairness via Stealthily Biased Sampling](https://mlsanity.com/paper/1901.08291/) (2019)
- [Algorithmic audits of algorithms, and the law](https://mlsanity.com/paper/2203.03711/) (2022)
- [Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability](https://mlsanity.com/paper/2406.01399/) (2024)
- [Robust ML Auditing using Prior Knowledge](https://mlsanity.com/paper/2505.04796/) (2025)
- [Explaining Machine Learning Models using Entropic Variable Projection](https://mlsanity.com/paper/1810.07924/) (2018)
- [Black-Box Audits for Group Distribution Shifts](https://mlsanity.com/paper/2209.03620/) (2022)
- [Leveraging Imperfect Sources to Detect Fairwashing in Black-Box Auditing](https://mlsanity.com/paper/2305.13883/) (2023)

## Built on by (most cited first, 2 of 2)

- [Evaluating Black-Box Vulnerabilities with Wasserstein-Constrained Data Perturbations](https://mlsanity.com/paper/2603.15867/) (2026)
- [The Evaluation Game: Beyond Static LLM Benchmarking](https://mlsanity.com/paper/2605.19377/) (2026)

## Abstract (arXiv)

The rapid deployment of AI systems in high-stakes domains, including those classified as high-risk under the The EU AI Act (Regulation (EU) 2024/1689), has intensified the need for reliable compliance auditing. For binary classifiers, regulatory risk assessment often relies on global fairness metrics such as the Disparate Impact ratio, widely used to evaluate potential discrimination. In typical auditing settings, the auditee provides a subset of its dataset to an auditor, while a supervisory authority may verify whether this subset is representative of the full underlying distribution. In this work, we investigate to what extent a malicious auditee can construct a fairness-compliant yet representative-looking sample from a non-compliant original distribution, thereby creating an illusion of fairness. We formalize this problem as a constrained distributional projection task and introduce mathematically grounded manipulation strategies based on entropic and optimal transport projections. These constructions characterize the minimal distributional shift required to satisfy fairness constraints. To counter such attacks, we formalize representativeness through distributional distance based statistical tests and systematically evaluate their ability to detect manipulated samples. Our analysis highlights the conditions under which fairness manipulation can remain statistically undetected and provides practical guidelines for strengthening supervisory verification. We validate our theoretical findings through experiments on standard tabular datasets for bias detection. Code is publicly available at https://github.com/ValentinLafargue/Inspection.