# Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent

arXiv: https://arxiv.org/abs/2206.02617 · HTML: https://mlsanity.com/paper/2206.02617/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 77 references that are in the map.
- First posted on arXiv 6 Jun 2022.

## Builds on (most cited first, 30 of 48)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Large Language Models Can Be Strong Differentially Private Learners](https://mlsanity.com/paper/2110.05679/) (2021)
- [Differentially Private Fine-tuning of Language Models](https://mlsanity.com/paper/2110.06500/) (2021)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences](https://mlsanity.com/paper/1807.01647/) (2018)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Unlocking High-Accuracy Differentially Private Image Classification through Scale](https://mlsanity.com/paper/2204.13650/) (2022)
- [Differentially Private Learning with Adaptive Clipping](https://mlsanity.com/paper/1905.03871/) (2019)
- [Auditing Differentially Private Machine Learning: How Private is Private SGD?](https://mlsanity.com/paper/2006.07709/) (2020)
- [R'enyi Differential Privacy of the Sampled Gaussian Mechanism](https://mlsanity.com/paper/1908.10530/) (2019)
- [Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning](https://mlsanity.com/paper/2101.04535/) (2021)
- [Numerical Composition of Differential Privacy](https://mlsanity.com/paper/2106.02848/) (2021)
- [Understanding Gradient Clipping in Private SGD: A Geometric Perspective](https://mlsanity.com/paper/2006.15429/) (2020)
- [Tempered Sigmoid Activations for Deep Learning with Differential Privacy](https://mlsanity.com/paper/2007.14191/) (2020)
- [Large-Scale Differentially Private BERT](https://mlsanity.com/paper/2108.01624/) (2021)

## Built on by (most cited first, 20 of 20)

- [PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs](https://mlsanity.com/paper/2406.02958/) (2024)
- [Have it your way: Individualized Privacy Assignment for DP-SGD](https://mlsanity.com/paper/2303.17046/) (2023)
- [Gradients Look Alike: Sensitivity is Often Overestimated in DP-SGD](https://mlsanity.com/paper/2307.00310/) (2023)
- [Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning](https://mlsanity.com/paper/2406.03519/) (2024)
- [Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learning](https://mlsanity.com/paper/2402.06674/) (2024)
- [Certification for Differentially Private Prediction in Gradient-Based Training](https://mlsanity.com/paper/2406.13433/) (2024)
- [Free Record-Level Privacy Risk Evaluation Through Artifact-Based Methods](https://mlsanity.com/paper/2411.05743/) (2024)
- [Crowding Out The Noise: Algorithmic Collective Action Under Differential Privacy](https://mlsanity.com/paper/2505.05707/) (2025)
- [Leveraging Per-Instance Privacy for Machine Unlearning](https://mlsanity.com/paper/2505.18786/) (2025)
- [Personalized DP-SGD using Sampling Mechanisms](https://mlsanity.com/paper/2305.15165/) (2023)
- [SoK: Memorisation in machine learning](https://mlsanity.com/paper/2311.03075/) (2023)
- [Laplace Sample Information: Data Informativeness Through a Bayesian Lens](https://mlsanity.com/paper/2505.15303/) (2025)
- [Your Privacy Depends on Others: Collusion Vulnerabilities in Individual Differential Privacy](https://mlsanity.com/paper/2601.12922/) (2026)
- [PACZero: PAC-Private Fine-Tuning of Language Models via Sign Quantization](https://mlsanity.com/paper/2605.06505/) (2026)
- [Personalized Privacy Amplification via Importance Sampling](https://mlsanity.com/paper/2307.10187/) (2023)
- [A Survey on Differential Privacy for SpatioTemporal Data in Transportation Research](https://mlsanity.com/paper/2407.15868/) (2024)
- [Controlled privacy leakage propagation throughout overlapping grouped learning](https://mlsanity.com/paper/2503.04054/) (2025)
- [PrivacyBench: Privacy Isn't Free in Hybrid Privacy-Preserving Vision Systems](https://mlsanity.com/paper/2602.18900/) (2026)
- [Technical Case Study of Privacy-Enhancing Technologies (PETs) for Public Health](https://mlsanity.com/paper/2603.13444/) (2026)
- [Performative Privacy: When Differential Privacy Maximizes Utility](https://mlsanity.com/paper/2608.28198/) (2026)

## Abstract (arXiv)

Differentially private stochastic gradient descent (DP-SGD) is the workhorse algorithm for recent advances in private deep learning. It provides a single privacy guarantee to all datapoints in the dataset. We propose output-specific (varepsilon,δ)-DP to characterize privacy guarantees for individual examples when releasing models trained by DP-SGD. We also design an efficient algorithm to investigate individual privacy across a number of datasets. We find that most examples enjoy stronger privacy guarantees than the worst-case bound. We further discover that the training loss and the privacy parameter of an example are well-correlated. This implies groups that are underserved in terms of model utility simultaneously experience weaker privacy guarantees. For example, on CIFAR-10, the average varepsilon of the class with the lowest test accuracy is 44.2% higher than that of the class with the highest accuracy.