# Not one but many Tradeoffs: Privacy Vs. Utility in Differentially Private Machine Learning

arXiv: https://arxiv.org/abs/2008.08807 · HTML: https://mlsanity.com/paper/2008.08807/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2020.

## Built on by (most cited first, 13 of 13)

- [PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments](https://mlsanity.com/paper/2104.14380/) (2021)
- [Evaluation Gaps in Machine Learning Practice](https://mlsanity.com/paper/2205.05256/) (2022)
- [Investigating Trade-offs in Utility, Fairness and Differential Privacy in Neural Networks](https://mlsanity.com/paper/2102.05975/) (2021)
- [Investigating the Impact of Dark Patterns on LLM-Based Web Agents](https://mlsanity.com/paper/2510.18113/) (2025)
- [DPMLBench: Holistic Evaluation of Differentially Private Machine Learning](https://mlsanity.com/paper/2305.05900/) (2023)
- [Bounding Information Leakage in Machine Learning](https://mlsanity.com/paper/2105.03875/) (2021)
- [Incentives in Private Collaborative Machine Learning](https://mlsanity.com/paper/2404.01676/) (2024)
- [DP-UTIL: Comprehensive Utility Analysis of Differential Privacy in Machine Learning](https://mlsanity.com/paper/2112.12998/) (2021)
- [Measuring What Matters: Connecting AI Ethics Evaluations to System Attributes, Hazards, and Harms](https://mlsanity.com/paper/2510.10339/) (2025)
- [Hierarchical Federated Learning with Privacy](https://mlsanity.com/paper/2206.05209/) (2022)
- [Does Privacy Always Harm Fairness? Data-Dependent Trade-offs via Chernoff Information Neural Estimation](https://mlsanity.com/paper/2601.13698/) (2026)
- [Rethinking Benchmarks for Differentially Private Image Classification](https://mlsanity.com/paper/2601.17189/) (2026)
- [Privacy-Utility Tradeoffs in Quantum Information Processing](https://mlsanity.com/paper/2602.10510/) (2026)

## Abstract (arXiv)

Data holders are increasingly seeking to protect their user's privacy, whilst still maximizing their ability to produce machine models with high quality predictions. In this work, we empirically evaluate various implementations of differential privacy (DP), and measure their ability to fend off real-world privacy attacks, in addition to measuring their core goal of providing accurate classifications. We establish an evaluation framework to ensure each of these implementations are fairly evaluated. Our selection of DP implementations add DP noise at different positions within the framework, either at the point of data collection/release, during updates while training of the model, or after training by perturbing learned model parameters. We evaluate each implementation across a range of privacy budgets, and datasets, each implementation providing the same mathematical privacy guarantees. By measuring the models' resistance to real world attacks of membership and attribute inference, and their classification accuracy. we determine which implementations provide the most desirable tradeoff between privacy and utility. We found that the number of classes of a given dataset is unlikely to influence where the privacy and utility tradeoff occurs. Additionally, in the scenario that high privacy constraints are required, perturbing input training data does not trade off as much utility, as compared to noise added later in the ML process.