# On the Vulnerability of Fairness Constrained Learning to Malicious Noise

arXiv: https://arxiv.org/abs/2307.11892 · HTML: https://mlsanity.com/paper/2307.11892/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 55 references that are in the map.
- First posted on arXiv 21 Jul 2023.

## Builds on (most cited first, 14 of 14)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Strategic Classification](https://mlsanity.com/paper/1506.06980/) (2015)
- [Outcome Indistinguishability](https://mlsanity.com/paper/2011.13426/) (2020)
- [Fairness in Credit Scoring: Assessment, Implementation and Profit Implications](https://mlsanity.com/paper/2103.01907/) (2021)
- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [Recovering from Biased Data: Can Fairness Constraints Improve Accuracy?](https://mlsanity.com/paper/1912.01094/) (2019)
- [Fair Classification with Adversarial Perturbations](https://mlsanity.com/paper/2106.05964/) (2021)
- [Robustly-reliable learners under poisoning attacks](https://mlsanity.com/paper/2203.04160/) (2022)
- [Minimax Group Fairness: Algorithms and Experiments](https://mlsanity.com/paper/2011.03108/) (2020)
- [FOCUS: Fairness via Agent-Awareness for Federated Learning on Heterogeneous Data](https://mlsanity.com/paper/2207.10265/) (2022)

## Built on by (most cited first, 3 of 3)

- [How Far Can Fairness Constraints Help Recover From Biased Data?](https://mlsanity.com/paper/2312.10396/) (2023)
- [On Optimal Steering to Achieve Exact Fairness](https://mlsanity.com/paper/2509.15759/) (2025)
- [A Comprehensive View of Fairness through Distributional Stability](https://mlsanity.com/paper/2609.37061/) (2026)

## Abstract (arXiv)

We consider the vulnerability of fairness-constrained learning to small amounts of malicious noise in the training data. Konstantinov and Lampert (2021) initiated the study of this question and presented negative results showing there exist data distributions where for several fairness constraints, any proper learner will exhibit high vulnerability when group sizes are imbalanced. Here, we present a more optimistic view, showing that if we allow randomized classifiers, then the landscape is much more nuanced. For example, for Demographic Parity we show we can incur only a Theta(α) loss in accuracy, where α is the malicious noise rate, matching the best possible even without fairness constraints. For Equal Opportunity, we show we can incur an O(√α) loss, and give a matching Omega(√α)lower bound. In contrast, Konstantinov and Lampert (2021) showed for proper learners the loss in accuracy for both notions is Omega(1). The key technical novelty of our work is how randomization can bypass simple "tricks" an adversary can use to amplify his power. We also consider additional fairness notions including Equalized Odds and Calibration. For these fairness notions, the excess accuracy clusters into three natural regimes O(α),O(√α) and O(1). These results provide a more fine-grained view of the sensitivity of fairness-constrained learning to adversarial noise in training data.