# Breaking Fair Binary Classification with Optimal Flipping Attacks

arXiv: https://arxiv.org/abs/2204.05472 · HTML: https://mlsanity.com/paper/2204.05472/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 12 Apr 2022.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Empirical Risk Minimization under Fairness Constraints](https://mlsanity.com/paper/1802.08626/) (2018)
- [Identifying and Correcting Label Bias in Machine Learning](https://mlsanity.com/paper/1901.04966/) (2019)
- [Stronger Data Poisoning Attacks Break Data Sanitization Defenses](https://mlsanity.com/paper/1811.00741/) (2018)
- [Fair Generative Modeling via Weak Supervision](https://mlsanity.com/paper/1910.12008/) (2019)
- [Certified Robustness to Label-Flipping Attacks via Randomized Smoothing](https://mlsanity.com/paper/2002.03018/) (2020)
- [FairBatch: Batch Selection for Model Fairness](https://mlsanity.com/paper/2012.01696/) (2020)
- [Beyond Parity: Fairness Objectives for Collaborative Filtering](https://mlsanity.com/paper/1705.08804/) (2017)
- [Robust Optimization for Fairness with Noisy Protected Groups](https://mlsanity.com/paper/2002.09343/) (2020)
- [Label Sanitization against Label Flipping Poisoning Attacks](https://mlsanity.com/paper/1803.00992/) (2018)
- [Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classification](https://mlsanity.com/paper/1906.05082/) (2019)
- [Poisoning Attacks on Algorithmic Fairness](https://mlsanity.com/paper/2004.07401/) (2020)
- [Two-Player Games for Efficient Non-Convex Constrained Optimization](https://mlsanity.com/paper/1804.06500/) (2018)
- [FR-Train: A Mutual Information-Based Approach to Fair and Robust Training](https://mlsanity.com/paper/2002.10234/) (2020)
- [Equalized odds postprocessing under imperfect group information](https://mlsanity.com/paper/1906.03284/) (2019)
- [Sample Selection for Fair and Robust Training](https://mlsanity.com/paper/2110.14222/) (2021)
- [Exacerbating Algorithmic Bias through Fairness Attacks](https://mlsanity.com/paper/2012.08723/) (2020)
- [Noise-tolerant fair classification](https://mlsanity.com/paper/1901.10837/) (2019)
- [Learning Fair Classifiers with Partially Annotated Group Labels](https://mlsanity.com/paper/2111.14581/) (2021)
- [On Adversarial Bias and the Robustness of Fair Machine Learning](https://mlsanity.com/paper/2006.08669/) (2020)
- [Mitigating Bias in Set Selection with Noisy Protected Attributes](https://mlsanity.com/paper/2011.04219/) (2020)
- [Fair Classification with Adversarial Perturbations](https://mlsanity.com/paper/2106.05964/) (2021)
- [Poisoning Attacks on Fair Machine Learning](https://mlsanity.com/paper/2110.08932/) (2021)

## Built on by (most cited first, 4 of 4)

- [Exploring Privacy and Fairness Risks in Sharing Diffusion Models: An Adversarial Perspective](https://mlsanity.com/paper/2402.18607/) (2024)
- [Fairness Hacking: The Malicious Practice of Shrouding Unfairness in Algorithms](https://mlsanity.com/paper/2311.06826/) (2023)
- [A Survey on Preserving Fairness Guarantees in Changing Environments](https://mlsanity.com/paper/2211.07530/) (2022)
- [Measuring Equality in Machine Learning Security Defenses: A Case Study in Speech Recognition](https://mlsanity.com/paper/2302.08973/) (2023)

## Abstract (arXiv)

Minimizing risk with fairness constraints is one of the popular approaches to learning a fair classifier. Recent works showed that this approach yields an unfair classifier if the training set is corrupted. In this work, we study the minimum amount of data corruption required for a successful flipping attack. First, we find lower/upper bounds on this quantity and show that these bounds are tight when the target model is the unique unconstrained risk minimizer. Second, we propose a computationally efficient data poisoning attack algorithm that can compromise the performance of fair learning algorithms.