# On the utility and protection of optimization with differential privacy and classic regularization techniques

arXiv: https://arxiv.org/abs/2209.03175 · HTML: https://mlsanity.com/paper/2209.03175/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 7 Sep 2022.

## Builds on (most cited first, 18 of 18)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Federated Learning with Differential Privacy: Algorithms and Performance Analysis](https://mlsanity.com/paper/1911.00222/) (2019)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks](https://mlsanity.com/paper/1911.07135/) (2019)
- [Machine Learning with Membership Privacy using Adversarial Regularization](https://mlsanity.com/paper/1807.05852/) (2018)
- [Concentrated Differential Privacy](https://mlsanity.com/paper/1603.01887/) (2016)
- [Privacy Preserving Machine Learning: Threats and Solutions](https://mlsanity.com/paper/1804.11238/) (2018)
- [Practical Blind Membership Inference Attack via Differential Comparisons](https://mlsanity.com/paper/2101.01341/) (2021)
- [Exploiting Explanations for Model Inversion Attacks](https://mlsanity.com/paper/2104.12669/) (2021)
- [Defending Model Inversion and Membership Inference Attacks via Prediction Purification](https://mlsanity.com/paper/2005.03915/) (2020)
- [To Drop or Not to Drop: Robustness, Consistency and Differential Privacy Properties of Dropout](https://mlsanity.com/paper/1503.02031/) (2015)
- [Differentially Private Dropout](https://mlsanity.com/paper/1712.01665/) (2017)

## Built on by (most cited first, 10 of 10)

- [Federated Knowledge Recycling: Privacy-Preserving Synthetic Data Sharing](https://mlsanity.com/paper/2407.20830/) (2024)
- [Synthetic Image Learning: Preserving Performance and Preventing Membership Inference Attacks](https://mlsanity.com/paper/2407.15526/) (2024)
- [Discriminative Adversarial Privacy: Balancing Accuracy and Membership Privacy in Neural Networks](https://mlsanity.com/paper/2306.03054/) (2023)
- [Differential Privacy: Gradient Leakage Attacks in Federated Learning Environments](https://mlsanity.com/paper/2510.23931/) (2025)
- [Revisiting the LiRA Membership Inference Attack Under Realistic Assumptions](https://mlsanity.com/paper/2603.07567/) (2026)
- [Your Image Generator Is Your New Private Dataset](https://mlsanity.com/paper/2504.04582/) (2025)
- [DISTINQT: A Distributed Privacy Aware Learning Framework for QoS Prediction for Future Mobile and Wireless Networks](https://mlsanity.com/paper/2401.10158/) (2024)
- [Differential Privacy Regularization: Protecting Training Data Through Loss Function Regularization](https://mlsanity.com/paper/2409.17144/) (2024)
- [Harnessing the Computing Continuum across Personalized Healthcare, Maintenance and Inspection, and Farming 4.0](https://mlsanity.com/paper/2403.14650/) (2024)
- [Fast Fourier Transform-Based Spectral and Temporal Gradient Filtering for Differential Privacy](https://mlsanity.com/paper/2505.04468/) (2025)

## Abstract (arXiv)

Nowadays, owners and developers of deep learning models must consider stringent privacy-preservation rules of their training data, usually crowd-sourced and retaining sensitive information. The most widely adopted method to enforce privacy guarantees of a deep learning model nowadays relies on optimization techniques enforcing differential privacy. According to the literature, this approach has proven to be a successful defence against several models' privacy attacks, but its downside is a substantial degradation of the models' performance. In this work, we compare the effectiveness of the differentially-private stochastic gradient descent (DP-SGD) algorithm against standard optimization practices with regularization techniques. We analyze the resulting models' utility, training performance, and the effectiveness of membership inference and model inversion attacks against the learned models. Finally, we discuss differential privacy's flaws and limits and empirically demonstrate the often superior privacy-preserving properties of dropout and l2-regularization.