# Poisoning Attacks on Algorithmic Fairness

arXiv: https://arxiv.org/abs/2004.07401 · HTML: https://mlsanity.com/paper/2004.07401/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2020.

## Built on by (most cited first, 30 of 59)

- [A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability](https://mlsanity.com/paper/2204.08570/) (2022)
- [Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective](https://mlsanity.com/paper/2112.06409/) (2021)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [Fairness in Graph Mining: A Survey](https://mlsanity.com/paper/2204.09888/) (2022)
- [Sample Selection for Fair and Robust Training](https://mlsanity.com/paper/2110.14222/) (2021)
- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Fairness-aware Adversarial Perturbation Towards Bias Mitigation for Deployed Deep Models](https://mlsanity.com/paper/2203.01584/) (2022)
- [Interpretable Data-Based Explanations for Fairness Debugging](https://mlsanity.com/paper/2112.09745/) (2021)
- [Machine Learning Security against Data Poisoning: Are We There Yet?](https://mlsanity.com/paper/2204.05986/) (2022)
- [Robust Conversational Agents against Imperceptible Toxicity Triggers](https://mlsanity.com/paper/2205.02392/) (2022)
- [Data Poisoning in Deep Learning: A Survey](https://mlsanity.com/paper/2503.22759/) (2025)
- [Poisoning Attacks on Fair Machine Learning](https://mlsanity.com/paper/2110.08932/) (2021)
- [Fairness-Aware PAC Learning from Corrupted Data](https://mlsanity.com/paper/2102.06004/) (2021)
- [AI Risk Management Should Incorporate Both Safety and Security](https://mlsanity.com/paper/2405.19524/) (2024)
- [Fooling Partial Dependence via Data Poisoning](https://mlsanity.com/paper/2105.12837/) (2021)
- [Algorithmic Fairness Verification with Graphical Models](https://mlsanity.com/paper/2109.09447/) (2021)
- [Improving Fair Training under Correlation Shifts](https://mlsanity.com/paper/2302.02323/) (2023)
- [Robust Fair Clustering: A Novel Fairness Attack and Defense Framework](https://mlsanity.com/paper/2210.01953/) (2022)
- [Adversarial Inter-Group Link Injection Degrades the Fairness of Graph Neural Networks](https://mlsanity.com/paper/2209.05957/) (2022)
- [Energy-Latency Attacks to On-Device Neural Networks via Sponge Poisoning](https://mlsanity.com/paper/2305.03888/) (2023)
- [Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples for Deep Learning Models](https://mlsanity.com/paper/2405.03869/) (2024)
- [Towards Multi-Objective Statistically Fair Federated Learning](https://mlsanity.com/paper/2201.09917/) (2022)
- [How Biased are Your Features?: Computing Fairness Influence Functions with Global Sensitivity Analysis](https://mlsanity.com/paper/2206.00667/) (2022)
- [Deceptive Fairness Attacks on Graphs via Meta Learning](https://mlsanity.com/paper/2310.15653/) (2023)
- [SoK: Unintended Interactions among Machine Learning Defenses and Risks](https://mlsanity.com/paper/2312.04542/) (2023)
- [Exploring Privacy and Fairness Risks in Sharing Diffusion Models: An Adversarial Perspective](https://mlsanity.com/paper/2402.18607/) (2024)
- [Machine Learning for Fraud Detection in E-Commerce: A Research Agenda](https://mlsanity.com/paper/2107.01979/) (2021)
- [Fairness Degrading Adversarial Attacks Against Clustering Algorithms](https://mlsanity.com/paper/2110.12020/) (2021)
- [Subverting Fair Image Search with Generative Adversarial Perturbations](https://mlsanity.com/paper/2205.02414/) (2022)
- [FairRoad: Achieving Fairness for Recommender Systems with Optimized Antidote Data](https://mlsanity.com/paper/2212.06750/) (2022)

## Abstract (arXiv)

Research in adversarial machine learning has shown how the performance of machine learning models can be seriously compromised by injecting even a small fraction of poisoning points into the training data. While the effects on model accuracy of such poisoning attacks have been widely studied, their potential effects on other model performance metrics remain to be evaluated. In this work, we introduce an optimization framework for poisoning attacks against algorithmic fairness, and develop a gradient-based poisoning attack aimed at introducing classification disparities among different groups in the data. We empirically show that our attack is effective not only in the white-box setting, in which the attacker has full access to the target model, but also in a more challenging black-box scenario in which the attacks are optimized against a substitute model and then transferred to the target model. We believe that our findings pave the way towards the definition of an entirely novel set of adversarial attacks targeting algorithmic fairness in different scenarios, and that investigating such vulnerabilities will help design more robust algorithms and countermeasures in the future.