# On the Privacy Risks of Algorithmic Fairness

arXiv: https://arxiv.org/abs/2011.03731 · HTML: https://mlsanity.com/paper/2011.03731/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2020.

## Built on by (most cited first, 30 of 82)

- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Low-Cost High-Power Membership Inference Attacks](https://mlsanity.com/paper/2312.03262/) (2023)
- [Machine Unlearning: Solutions and Challenges](https://mlsanity.com/paper/2308.07061/) (2023)
- [Bias Testing and Mitigation in LLM-based Code Generation](https://mlsanity.com/paper/2309.14345/) (2023)
- [SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning](https://mlsanity.com/paper/2212.10986/) (2022)
- [Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities](https://mlsanity.com/paper/2102.04257/) (2021)
- [Disparate Impact in Differential Privacy from Gradient Misalignment](https://mlsanity.com/paper/2206.07737/) (2022)
- [Pre-trained Perceptual Features Improve Differentially Private Image Generation](https://mlsanity.com/paper/2205.12900/) (2022)
- [Achilles' Heels: Vulnerable Record Identification in Synthetic Data Publishing](https://mlsanity.com/paper/2306.10308/) (2023)
- [Privacy and Fairness in Federated Learning: on the Perspective of Trade-off](https://mlsanity.com/paper/2306.14123/) (2023)
- [The privacy issue of counterfactual explanations: explanation linkage attacks](https://mlsanity.com/paper/2210.12051/) (2022)
- [How unfair is private learning ?](https://mlsanity.com/paper/2206.03985/) (2022)
- [Differential Privacy has Bounded Impact on Fairness in Classification](https://mlsanity.com/paper/2210.16242/) (2022)
- [Unraveling Privacy Risks of Individual Fairness in Graph Neural Networks](https://mlsanity.com/paper/2301.12951/) (2023)
- [Unlocking Accuracy and Fairness in Differentially Private Image Classification](https://mlsanity.com/paper/2308.10888/) (2023)
- [Demographic-Reliant Algorithmic Fairness: Characterizing the Risks of Demographic Data Collection in the Pursuit of Fairness](https://mlsanity.com/paper/2205.01038/) (2022)
- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [Conflicting Interactions Among Protection Mechanisms for Machine Learning Models](https://mlsanity.com/paper/2207.01991/) (2022)
- [The Impact of Differential Privacy on Group Disparity Mitigation](https://mlsanity.com/paper/2203.02745/) (2022)
- [SoK: Machine Learning Governance](https://mlsanity.com/paper/2109.10870/) (2021)
- [Exploiting Fairness to Enhance Sensitive Attributes Reconstruction](https://mlsanity.com/paper/2209.01215/) (2022)
- [Learning with Impartiality to Walk on the Pareto Frontier of Fairness, Privacy, and Utility](https://mlsanity.com/paper/2302.09183/) (2023)
- [Fair NLP Models with Differentially Private Text Encoders](https://mlsanity.com/paper/2205.06135/) (2022)
- [On the Alignment of Group Fairness with Attribute Privacy](https://mlsanity.com/paper/2211.10209/) (2022)
- [An Empirical Analysis of Fairness Notions under Differential Privacy](https://mlsanity.com/paper/2302.02910/) (2023)
- [Counterfactual Concept Bottleneck Models](https://mlsanity.com/paper/2402.01408/) (2024)
- [Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learning](https://mlsanity.com/paper/2402.06674/) (2024)
- [When Fairness Meets Privacy: Fair Classification with Semi-Private Sensitive Attributes](https://mlsanity.com/paper/2207.08336/) (2022)
- [Data Privacy and Trustworthy Machine Learning](https://mlsanity.com/paper/2209.06529/) (2022)

## Abstract (arXiv)

Algorithmic fairness and privacy are essential pillars of trustworthy machine learning. Fair machine learning aims at minimizing discrimination against protected groups by, for example, imposing a constraint on models to equalize their behavior across different groups. This can subsequently change the influence of training data points on the fair model, in a disproportionate way. We study how this can change the information leakage of the model about its training data. We analyze the privacy risks of group fairness (e.g., equalized odds) through the lens of membership inference attacks: inferring whether a data point is used for training a model. We show that fairness comes at the cost of privacy, and this cost is not distributed equally: the information leakage of fair models increases significantly on the unprivileged subgroups, which are the ones for whom we need fair learning. We show that the more biased the training data is, the higher the privacy cost of achieving fairness for the unprivileged subgroups will be. We provide comprehensive empirical analysis for general machine learning algorithms.