# BadFair: Backdoored Fairness Attacks with Group-conditioned Triggers

arXiv: https://arxiv.org/abs/2410.17492 · HTML: https://mlsanity.com/paper/2410.17492/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 49 references that are in the map.
- First posted on arXiv 23 Oct 2024.

## Builds on (most cited first, 30 of 33)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Poisoning Language Models During Instruction Tuning](https://mlsanity.com/paper/2305.00944/) (2023)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger](https://mlsanity.com/paper/2105.12400/) (2021)
- [To be Robust or to be Fair: Towards Fairness in Adversarial Training](https://mlsanity.com/paper/2010.06121/) (2020)
- [Fairness in Deep Learning: A Computational Perspective](https://mlsanity.com/paper/1908.08843/) (2019)
- [Subpopulation Data Poisoning Attacks](https://mlsanity.com/paper/2006.14026/) (2020)
- [Poisoning Attacks on Algorithmic Fairness](https://mlsanity.com/paper/2004.07401/) (2020)
- [Challenges for Toxic Comment Classification: An In-Depth Error Analysis](https://mlsanity.com/paper/1809.07572/) (2018)
- [Exacerbating Algorithmic Bias through Fairness Attacks](https://mlsanity.com/paper/2012.08723/) (2020)
- [Fairness-aware Adversarial Perturbation Towards Bias Mitigation for Deployed Deep Models](https://mlsanity.com/paper/2203.01584/) (2022)
- [On Adversarial Bias and the Robustness of Fair Machine Learning](https://mlsanity.com/paper/2006.08669/) (2020)
- [RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR](https://mlsanity.com/paper/2111.11665/) (2021)
- [TrojText: Test-time Invisible Textual Trojan Insertion](https://mlsanity.com/paper/2303.02242/) (2023)
- [Estimating and Improving Fairness with Adversarial Learning](https://mlsanity.com/paper/2103.04243/) (2021)
- [Backdoor Learning on Sequence to Sequence Models](https://mlsanity.com/paper/2305.02424/) (2023)
- [Poisoning Attacks on Fair Machine Learning](https://mlsanity.com/paper/2110.08932/) (2021)
- [Early Prediction of Alzheimer's Disease Dementia Based on Baseline Hippocampal MRI and 1-Year Follow-Up Cognitive Measures Using Deep Recurrent Neural Networks](https://mlsanity.com/paper/1901.01451/) (2019)
- [A benchmark for toxic comment classification on Civil Comments dataset](https://mlsanity.com/paper/2301.11125/) (2023)
- [ESTAS: Effective and Stable Trojan Attacks in Self-supervised Encoders with One Target Unlabelled Sample](https://mlsanity.com/paper/2211.10908/) (2022)

## Built on by (most cited first, 4 of 4)

- [Your RAG is Unfair: Exposing Fairness Vulnerabilities in Retrieval-Augmented Generation via Backdoor Attacks](https://mlsanity.com/paper/2509.22486/) (2025)
- [Factuality Beyond Coherence: Evaluating LLM Watermarking Methods for Medical Texts](https://mlsanity.com/paper/2509.07755/) (2025)
- [In-Context Bias Propagation in LLM-Based Tabular Data Generation](https://mlsanity.com/paper/2506.09630/) (2025)
- [Anchoring Bias: A Persistent Fairness Backdoor Attack against MLLMs under Continual Learning](https://mlsanity.com/paper/2608.21577/) (2026)

## Abstract (arXiv)

Attacking fairness is crucial because compromised models can introduce biased outcomes, undermining trust and amplifying inequalities in sensitive applications like hiring, healthcare, and law enforcement. This highlights the urgent need to understand how fairness mechanisms can be exploited and to develop defenses that ensure both fairness and robustness. We introduce BadFair, a novel backdoored fairness attack methodology. BadFair stealthily crafts a model that operates with accuracy and fairness under regular conditions but, when activated by certain triggers, discriminates and produces incorrect results for specific groups. This type of attack is particularly stealthy and dangerous, as it circumvents existing fairness detection methods, maintaining an appearance of fairness in normal use. Our findings reveal that BadFair achieves a more than 85% attack success rate in attacks aimed at target groups on average while only incurring a minimal accuracy loss. Moreover, it consistently exhibits a significant discrimination score, distinguishing between pre-defined target and non-target attacked groups across various datasets and models.