# How Well Can Differential Privacy Be Audited in One Run?

arXiv: https://arxiv.org/abs/2503.07199 · HTML: https://mlsanity.com/paper/2503.07199/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 10 Mar 2025.

## Builds on (most cited first, 10 of 10)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Auditing Differentially Private Machine Learning: How Private is Private SGD?](https://mlsanity.com/paper/2006.07709/) (2020)
- [Privacy Auditing with One (1) Training Run](https://mlsanity.com/paper/2305.08846/) (2023)
- [Antipodes of Label Differential Privacy: PATE and ALIBI](https://mlsanity.com/paper/2106.03408/) (2021)
- [Detecting Violations of Differential Privacy](https://mlsanity.com/paper/1805.10277/) (2018)
- [Evaluations of Machine Learning Privacy Defenses are Misleading](https://mlsanity.com/paper/2404.17399/) (2024)
- [Bayesian Estimation of Differential Privacy](https://mlsanity.com/paper/2206.05199/) (2022)
- [Auditing f-Differential Privacy in One Run](https://mlsanity.com/paper/2410.22235/) (2024)
- [Privacy Audit as Bits Transmission: (Im)possibilities for Audit by One Run](https://mlsanity.com/paper/2501.17750/) (2025)

## Built on by (most cited first, 9 of 9)

- [Optimizing Canaries for Privacy Auditing with Metagradient Descent](https://mlsanity.com/paper/2507.15836/) (2025)
- [The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditing](https://mlsanity.com/paper/2506.16666/) (2025)
- [A Unified Framework for Adversary-Aware Differential Privacy Bounds](https://mlsanity.com/paper/2507.08158/) (2025)
- [Causal Evaluation of Membership Inference Attacks](https://mlsanity.com/paper/2602.02819/) (2026)
- [On Reliability of Membership Inference Vulnerability Evaluation](https://mlsanity.com/paper/2605.25819/) (2026)
- [Detectability in Diversity: Improved Canary Crafting for Privacy Auditing in One Run](https://mlsanity.com/paper/2605.27292/) (2026)
- [Validity Threats for Foundation Model Research](https://mlsanity.com/paper/2606.05029/) (2026)
- [Let's Ask Gauss: Improved One-Run Privacy Auditing](https://mlsanity.com/paper/2606.12733/) (2026)
- [Calibrating One-Round Membership Inference with Neighbors](https://mlsanity.com/paper/2609.36331/) (2026)

## Abstract (arXiv)

Recent methods for auditing the privacy of machine learning algorithms have improved computational efficiency by simultaneously intervening on multiple training examples in a single training run. Steinke et al. (2024) prove that one-run auditing indeed lower bounds the true privacy parameter of the audited algorithm, and give impressive empirical results. Their work leaves open the question of how precisely one-run auditing can uncover the true privacy parameter of an algorithm, and how that precision depends on the audited algorithm. In this work, we characterize the maximum achievable efficacy of one-run auditing and show that the key barrier to its efficacy is interference between the observable effects of different data elements. We present new conceptual approaches to minimize this barrier, towards improving the performance of one-run auditing of real machine learning algorithms.