# Bayesian Estimation of Differential Privacy

arXiv: https://arxiv.org/abs/2206.05199 · HTML: https://mlsanity.com/paper/2206.05199/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 10 Jun 2022.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 18 of 18)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [The Composition Theorem for Differential Privacy](https://mlsanity.com/paper/1311.0776/) (2013)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Auditing Differentially Private Machine Learning: How Private is Private SGD?](https://mlsanity.com/paper/2006.07709/) (2020)
- [Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning](https://mlsanity.com/paper/2101.04535/) (2021)
- [Numerical Composition of Differential Privacy](https://mlsanity.com/paper/2106.02848/) (2021)
- [Analyzing Information Leakage of Updates to Natural Language Models](https://mlsanity.com/paper/1912.07942/) (2019)
- [Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descent](https://mlsanity.com/paper/2102.05855/) (2021)
- [Antipodes of Label Differential Privacy: PATE and ALIBI](https://mlsanity.com/paper/2106.03408/) (2021)
- [Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFT](https://mlsanity.com/paper/2006.07134/) (2020)
- [Detecting Violations of Differential Privacy](https://mlsanity.com/paper/1805.10277/) (2018)
- [Disparate Vulnerability to Membership Inference Attacks](https://mlsanity.com/paper/1906.00389/) (2019)
- [Investigating Membership Inference Attacks under Data Dependencies](https://mlsanity.com/paper/2010.12112/) (2020)
- [That which we call private](https://mlsanity.com/paper/1908.03566/) (2019)

## Built on by (most cited first, 30 of 33)

- [Privacy Auditing with One (1) Training Run](https://mlsanity.com/paper/2305.08846/) (2023)
- [Measuring Forgetting of Memorized Training Examples](https://mlsanity.com/paper/2207.00099/) (2022)
- [TAPAS: a Toolbox for Adversarial Privacy Auditing of Synthetic Data](https://mlsanity.com/paper/2211.06550/) (2022)
- [SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning](https://mlsanity.com/paper/2212.10986/) (2022)
- [One-shot Empirical Privacy Estimation for Federated Learning](https://mlsanity.com/paper/2302.03098/) (2023)
- [Label Poisoning is All You Need](https://mlsanity.com/paper/2310.18933/) (2023)
- [Gaussian Membership Inference Privacy](https://mlsanity.com/paper/2306.07273/) (2023)
- [Nearly Tight Black-Box Auditing of Differentially Private Machine Learning](https://mlsanity.com/paper/2405.14106/) (2024)
- [Unleashing the Power of Randomization in Auditing Differentially Private ML](https://mlsanity.com/paper/2305.18447/) (2023)
- [The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text](https://mlsanity.com/paper/2502.14921/) (2025)
- [PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining](https://mlsanity.com/paper/2402.09477/) (2024)
- [Shake to Leak: Fine-tuning Diffusion Models Can Amplify the Generative Privacy Risk](https://mlsanity.com/paper/2403.09450/) (2024)
- [The Elusive Pursuit of Reproducing PATE-GAN: Benchmarking, Auditing, Debugging](https://mlsanity.com/paper/2406.13985/) (2024)
- [Gradients Look Alike: Sensitivity is Often Overestimated in DP-SGD](https://mlsanity.com/paper/2307.00310/) (2023)
- [The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD](https://mlsanity.com/paper/2410.06186/) (2024)
- [How Well Can Differential Privacy Be Audited in One Run?](https://mlsanity.com/paper/2503.07199/) (2025)
- [Free Record-Level Privacy Risk Evaluation Through Artifact-Based Methods](https://mlsanity.com/paper/2411.05743/) (2024)
- [A Note On Interpreting Canary Exposure](https://mlsanity.com/paper/2306.00133/) (2023)
- [Auditing Differential Privacy Guarantees Using Density Estimation](https://mlsanity.com/paper/2406.04827/) (2024)
- [To Shuffle or not to Shuffle: Auditing DP-SGD with Shuffling](https://mlsanity.com/paper/2411.10614/) (2024)
- [Revisiting Differentially Private Hyper-parameter Tuning](https://mlsanity.com/paper/2402.13087/) (2024)
- [The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditing](https://mlsanity.com/paper/2506.16666/) (2025)
- [ContextLeak: Auditing Leakage in Private In-Context Learning Methods](https://mlsanity.com/paper/2512.16059/) (2025)
- [Closed-Form Bounds for DP-SGD against Record-level Inference](https://mlsanity.com/paper/2402.14397/) (2024)
- [Tight Auditing of Differential Privacy in MST and AIM](https://mlsanity.com/paper/2604.18352/) (2026)
- [Adversarial Sample-Based Approach for Tighter Privacy Auditing in Final Model-Only Scenarios](https://mlsanity.com/paper/2412.01756/) (2024)
- [On the Fairness of Privacy Protection: Measuring and Mitigating the Disparity of Group Privacy Risks for Differentially Private Machine Learning](https://mlsanity.com/paper/2510.09114/) (2025)
- [Observational Auditing of Label Privacy](https://mlsanity.com/paper/2511.14084/) (2025)
- [Beyond Membership: Limitations of Add/Remove Adjacency in Differential Privacy](https://mlsanity.com/paper/2511.21804/) (2025)
- [Exposing and Defending Membership Leakage in Vulnerability Prediction Models](https://mlsanity.com/paper/2512.08291/) (2025)

## Abstract (arXiv)

Algorithms such as Differentially Private SGD enable training machine learning models with formal privacy guarantees. However, there is a discrepancy between the protection that such algorithms guarantee in theory and the protection they afford in practice. An emerging strand of work empirically estimates the protection afforded by differentially private training as a confidence interval for the privacy budget varepsilon spent on training a model. Existing approaches derive confidence intervals for varepsilon from confidence intervals for the false positive and false negative rates of membership inference attacks. Unfortunately, obtaining narrow high-confidence intervals for ε using this method requires an impractically large sample size and training as many models as samples. We propose a novel Bayesian method that greatly reduces sample size, and adapt and validate a heuristic to draw more than one sample per trained model. Our Bayesian method exploits the hypothesis testing interpretation of differential privacy to obtain a posterior for varepsilon (not just a confidence interval) from the joint posterior of the false positive and false negative rates of membership inference attacks. For the same sample size and confidence, we derive confidence intervals for varepsilon around 40% narrower than prior work. The heuristic, which we adapt from label-only DP, can be used to further reduce the number of trained models needed to get enough samples by up to 2 orders of magnitude.