# Poisoning Attacks on Fair Machine Learning

arXiv: https://arxiv.org/abs/2110.08932 · HTML: https://mlsanity.com/paper/2110.08932/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 17 Oct 2021.

## Builds on (most cited first, 17 of 17)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Adversarial Examples: Attacks and Defenses for Deep Learning](https://mlsanity.com/paper/1712.07107/) (2017)
- [Certified Defenses for Data Poisoning Attacks](https://mlsanity.com/paper/1706.03691/) (2017)
- [Adversarial Attacks and Defences: A Survey](https://mlsanity.com/paper/1810.00069/) (2018)
- [Empirical Risk Minimization under Fairness Constraints](https://mlsanity.com/paper/1802.08626/) (2018)
- [Stronger Data Poisoning Attacks Break Data Sanitization Defenses](https://mlsanity.com/paper/1811.00741/) (2018)
- [PC-Fairness: A Unified Framework for Measuring Causality-based Fairness](https://mlsanity.com/paper/1910.12586/) (2019)
- [Poisoning Attacks on Algorithmic Fairness](https://mlsanity.com/paper/2004.07401/) (2020)
- [FR-Train: A Mutual Information-Based Approach to Fair and Robust Training](https://mlsanity.com/paper/2002.10234/) (2020)
- [Exacerbating Algorithmic Bias through Fairness Attacks](https://mlsanity.com/paper/2012.08723/) (2020)
- [The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learning](https://mlsanity.com/paper/1907.04472/) (2019)
- [A Distributionally Robust Approach to Fair Classification](https://mlsanity.com/paper/2007.09530/) (2020)
- [On Adversarial Bias and the Robustness of Fair Machine Learning](https://mlsanity.com/paper/2006.08669/) (2020)
- [Poisoning Attacks with Generative Adversarial Nets](https://mlsanity.com/paper/1906.07773/) (2019)

## Built on by (most cited first, 19 of 19)

- [A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability](https://mlsanity.com/paper/2204.08570/) (2022)
- [Data Poisoning in Deep Learning: A Survey](https://mlsanity.com/paper/2503.22759/) (2025)
- [Deceptive Fairness Attacks on Graphs via Meta Learning](https://mlsanity.com/paper/2310.15653/) (2023)
- [SoK: Unintended Interactions among Machine Learning Defenses and Risks](https://mlsanity.com/paper/2312.04542/) (2023)
- [Exploring Privacy and Fairness Risks in Sharing Diffusion Models: An Adversarial Perspective](https://mlsanity.com/paper/2402.18607/) (2024)
- [Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond](https://mlsanity.com/paper/2503.07501/) (2025)
- [Breaking Fair Binary Classification with Optimal Flipping Attacks](https://mlsanity.com/paper/2204.05472/) (2022)
- [Fairness-aware Regression Robust to Adversarial Attacks](https://mlsanity.com/paper/2211.04449/) (2022)
- [Adversarial Attacks on Fairness of Graph Neural Networks](https://mlsanity.com/paper/2310.13822/) (2023)
- [BadFair: Backdoored Fairness Attacks with Group-conditioned Triggers](https://mlsanity.com/paper/2410.17492/) (2024)
- [Beyond Detection: Unveiling Fairness Vulnerabilities in Abusive Language Models](https://mlsanity.com/paper/2311.09428/) (2023)
- [TrojFair: Trojan Fairness Attacks](https://mlsanity.com/paper/2312.10508/) (2023)
- [Multigroup Robustness](https://mlsanity.com/paper/2405.00614/) (2024)
- [HINT: Healthy Influential-Noise based Training to Defend against Data Poisoning Attacks](https://mlsanity.com/paper/2309.08549/) (2023)
- [Attacks on fairness in Federated Learning](https://mlsanity.com/paper/2311.12715/) (2023)
- [FairSAM: Fair Classification on Corrupted Image Data Through Sharpness-Aware Minimization](https://mlsanity.com/paper/2503.22934/) (2025)
- [In-Context Bias Propagation in LLM-Based Tabular Data Generation](https://mlsanity.com/paper/2506.09630/) (2025)
- [Mole Recruitment: Poisoning of Image Classifiers via Selective Batch Sampling](https://mlsanity.com/paper/2303.17080/) (2023)
- [Adversarial Bias: Data Poisoning Attacks on Fairness](https://mlsanity.com/paper/2511.08331/) (2025)

## Abstract (arXiv)

Both fair machine learning and adversarial learning have been extensively studied. However, attacking fair machine learning models has received less attention. In this paper, we present a framework that seeks to effectively generate poisoning samples to attack both model accuracy and algorithmic fairness. Our attacking framework can target fair machine learning models trained with a variety of group based fairness notions such as demographic parity and equalized odds. We develop three online attacks, adversarial sampling , adversarial labeling, and adversarial feature modification. All three attacks effectively and efficiently produce poisoning samples via sampling, labeling, or modifying a fraction of training data in order to reduce the test accuracy. Our framework enables attackers to flexibly adjust the attack's focus on prediction accuracy or fairness and accurately quantify the impact of each candidate point to both accuracy loss and fairness violation, thus producing effective poisoning samples. Experiments on two real datasets demonstrate the effectiveness and efficiency of our framework.