Data to 7 Oct 2026

Differential Privacy Has Disparate Impact on Model Accuracy

arXiv 1905.12101 ↗First posted 28 May 2019cs.LG cs.CR stat.MLcited by 316

Not on the map. Why?

Abstract (arXiv)

Differential privacy (DP) is a popular mechanism for training machine learning models with bounded leakage about the presence of specific points in the training data. The cost of differential privacy is a reduction in the model's accuracy. We demonstrate that in the neural networks trained using differentially private stochastic gradient descent (DP-SGD), this cost is not borne equally: accuracy of DP models drops much more for the underrepresented classes and subgroups. For example, a gender classification model trained using DP-SGD exhibits much lower accuracy for black faces than for white faces. Critically, this gap is bigger in the DP model than in the non-DP model, i.e., if the original model is unfair, the unfairness becomes worse once DP is applied. We demonstrate this effect for a variety of tasks and models, including sentiment analysis of text and image classification. We then explain why DP training mechanisms such as gradient clipping and noise addition have disproportionate effect on the underrepresented and more complex subgroups, resulting in a disparate reduction of model accuracy.

Key facts. Cited by 316 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 45; 2022: 61; 2023: 75; 2024: 61; 2025: 45; 2026: 29.

316 papers.

Sort:

By area:

2021A Field Guide to Federated Optimizationcited by 312
2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
2022Datamodels: Predicting Predictions from Training Datacited by 186
2022The Privacy Onion Effect: Memorization is Relativecited by 128
2023Machine Learning for Synthetic Data Generation: A Reviewcited by 118
2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
2021PPFL: Privacy-preserving Federated Learning with Trusted Execution Environmentscited by 93
2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
2023Revisiting Gradient Clipping: Stochastic bias and tight convergence guaranteescited by 83
2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 81
2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76

38 more

2022Fishing for User Data in Large-Batch Federated Learning via Gradient Magnificationcited by 72
2023Membership Inference Attacks against Synthetic Data through Overfitting Detectioncited by 66
2021Privacy-Preserving Machine Learning: Methods, Challenges and Directionscited by 58
2022Mixed Differential Privacy in Computer Visioncited by 53
2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 52
2022How Different Groups Prioritize Ethical Values for Responsible AIcited by 49
2021Robin Hood and Matthew Effects: Differential Privacy Has Disparate Impact on Synthetic Datacited by 48
2022Differential Privacy and Fairness in Decisions and Learning Tasks: A Surveycited by 47
2022Do Gradient Inversion Attacks Make Federated Learning Unsafe?cited by 44
2021Node-Level Differentially Private Graph Neural Networkscited by 43
2022Pruning has a disparate impact on model accuracycited by 43
2024How Does Quantization Affect Multilingual LLMs?cited by 41
2021Optimality and Stability in Federated Learning: A Game-theoretic Approachcited by 40
2021Enforcing fairness in private federated learning via the modified method of differential multiplierscited by 38
2021On the Convergence and Calibration of Deep Learning with Differential Privacycited by 36
2021Membership Inference on Word Embedding and Beyondcited by 34
2023Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directionscited by 33
2021DP-SGD vs PATE: Which Has Less Disparate Impact on Model Accuracy?cited by 32
2022Disparate Impact in Differential Privacy from Gradient Misalignmentcited by 31
2021Quantum machine learning with differential privacycited by 30
2023Identifying and Mitigating Privacy Risks Stemming from Language Models: A Surveycited by 30
2021Privacy Regularization: Joint Privacy-Utility Optimization in Language Modelscited by 29
2022Amplifying Membership Exposure via Data Poisoningcited by 29
2021The Rich Get Richer: Disparate Impact of Semi-Supervised Learningcited by 28
2022Pre-trained Perceptual Features Improve Differentially Private Image Generationcited by 28
2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
2023Privacy and Fairness in Federated Learning: on the Perspective of Trade-offcited by 26
2023Fair Machine Unlearning: Data Removal while Mitigating Disparitiescited by 25
2024Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metricscited by 25
2021Accuracy, Interpretability, and Differential Privacy via Explainable Boostingcited by 24
2021PEARL: Data Synthesis via Private Embeddings and Adversarial Reconstruction Learningcited by 23
2021Fair Classification with Adversarial Perturbationscited by 23
2021RoFL: Robustness of Secure Federated Learningcited by 23
2023A Linear Reconstruction Approach for Attribute Inference Attacks against Synthetic Datacited by 23
2022FLAIR: Federated Learning Annotated Image Repositorycited by 22
2023No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device MLcited by 21
2024The Better Angels of Machine Personality: How Personality Relates to LLM Safetycited by 21
2022Individual Privacy Accounting for Differentially Private Stochastic Gradient Descentcited by 20

50 most cited of 316 All 316 (JSON)