# Fair Classification with Adversarial Perturbations

arXiv: https://arxiv.org/abs/2106.05964 · HTML: https://mlsanity.com/paper/2106.05964/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 77 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 28 of 28)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [Delayed Impact of Fair Machine Learning](https://mlsanity.com/paper/1803.04383/) (2018)
- [Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees](https://mlsanity.com/paper/1806.06055/) (2018)
- [Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved](https://mlsanity.com/paper/1811.11154/) (2018)
- [Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination](https://mlsanity.com/paper/1906.00285/) (2019)
- [Robust Optimization for Fairness with Noisy Protected Groups](https://mlsanity.com/paper/2002.09343/) (2020)
- [A Confidence-Based Approach for Balancing Fairness and Accuracy](https://mlsanity.com/paper/1601.05764/) (2016)
- [Classification with Asymmetric Label Noise: Consistency and Maximal Denoising](https://mlsanity.com/paper/1303.1208/) (2013)
- [Satisfying Real-world Goals with Dataset Constraints](https://mlsanity.com/paper/1606.07558/) (2016)
- [Fair Classification with Group-Dependent Label Noise](https://mlsanity.com/paper/2011.00379/) (2020)
- [Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning](https://mlsanity.com/paper/1902.04783/) (2019)
- [Recovering from Biased Data: Can Fairness Constraints Improve Accuracy?](https://mlsanity.com/paper/1912.01094/) (2019)
- [Equalized odds postprocessing under imperfect group information](https://mlsanity.com/paper/1906.03284/) (2019)
- [Noise-tolerant fair classification](https://mlsanity.com/paper/1901.10837/) (2019)
- [Proxy Fairness](https://mlsanity.com/paper/1806.11212/) (2018)
- [Understanding Unequal Gender Classification Accuracy from Face Images](https://mlsanity.com/paper/1812.00099/) (2018)
- [Fair Classification with Noisy Protected Attributes: A Framework with Provable Guarantees](https://mlsanity.com/paper/2006.04778/) (2020)

## Built on by (most cited first, 23 of 23)

- [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022)
- [Fair Ranking with Noisy Protected Attributes](https://mlsanity.com/paper/2211.17067/) (2022)
- [Robust Fair Clustering: A Novel Fairness Attack and Defense Framework](https://mlsanity.com/paper/2210.01953/) (2022)
- [Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach](https://mlsanity.com/paper/2303.03300/) (2023)
- [Input-agnostic Certified Group Fairness via Gaussian Parameter Smoothing](https://mlsanity.com/paper/2206.11423/) (2022)
- [Data Optimization in Deep Learning: A Survey](https://mlsanity.com/paper/2310.16499/) (2023)
- [Subverting Fair Image Search with Generative Adversarial Perturbations](https://mlsanity.com/paper/2205.02414/) (2022)
- [Breaking Fair Binary Classification with Optimal Flipping Attacks](https://mlsanity.com/paper/2204.05472/) (2022)
- [Fairness without Demographics through Learning Graph of Gradients](https://mlsanity.com/paper/2412.03706/) (2024)
- [Distributionally Robust Data Join](https://mlsanity.com/paper/2202.05797/) (2022)
- [On the Vulnerability of Fairness Constrained Learning to Malicious Noise](https://mlsanity.com/paper/2307.11892/) (2023)
- [A Survey on Preserving Fairness Guarantees in Changing Environments](https://mlsanity.com/paper/2211.07530/) (2022)
- [Group Fairness with Uncertainty in Sensitive Attributes](https://mlsanity.com/paper/2302.08077/) (2023)
- [Sampling Individually-Fair Rankings that are Always Group Fair](https://mlsanity.com/paper/2306.11964/) (2023)
- [Fair Classification with Partial Feedback: An Exploration-Based Data Collection Approach](https://mlsanity.com/paper/2402.11338/) (2024)
- [Towards Fair Classification against Poisoning Attacks](https://mlsanity.com/paper/2210.09503/) (2022)
- [Towards Better Fairness-Utility Trade-off: A Comprehensive Measurement-Based Reinforcement Learning Framework](https://mlsanity.com/paper/2307.11379/) (2023)
- [FLEA: Provably Robust Fair Multisource Learning from Unreliable Training Data](https://mlsanity.com/paper/2106.11732/) (2021)
- [Measuring Equality in Machine Learning Security Defenses: A Case Study in Speech Recognition](https://mlsanity.com/paper/2302.08973/) (2023)
- [On the Cause of Unfairness: A Training Sample Perspective](https://mlsanity.com/paper/2306.17828/) (2023)
- [Fairness Risks for Group-conditionally Missing Demographics](https://mlsanity.com/paper/2402.13393/) (2024)
- [Price of Fairness in Short-Term and Long-Term Algorithmic Selections](https://mlsanity.com/paper/2605.06227/) (2026)
- [Fairness in multi-class multi-group classification problems via contextial coherent risk measures](https://mlsanity.com/paper/2608.30223/) (2026)

## Abstract (arXiv)

We study fair classification in the presence of an omniscient adversary that, given an η, is allowed to choose an arbitrary η-fraction of the training samples and arbitrarily perturb their protected attributes. The motivation comes from settings in which protected attributes can be incorrect due to strategic misreporting, malicious actors, or errors in imputation; and prior approaches that make stochastic or independence assumptions on errors may not satisfy their guarantees in this adversarial setting. Our main contribution is an optimization framework to learn fair classifiers in this adversarial setting that comes with provable guarantees on accuracy and fairness. Our framework works with multiple and non-binary protected attributes, is designed for the large class of linear-fractional fairness metrics, and can also handle perturbations besides protected attributes. We prove near-tightness of our framework's guarantees for natural hypothesis classes: no algorithm can have significantly better accuracy and any algorithm with better fairness must have lower accuracy. Empirically, we evaluate the classifiers produced by our framework for statistical rate on real-world and synthetic datasets for a family of adversaries.