mlsanityData to 14 Oct 2026

Sequentially Auditing Differential Privacy

arXiv 2509.07055 ↗First posted 8 Sep 2025cs.CR cs.LG stat.MEcited by 5

Not on the map. Why?

Abstract (arXiv)

We propose a practical sequential test for auditing differential privacy guarantees of black-box mechanisms. The test processes streams of mechanisms' outputs providing anytime-valid inference while controlling Type I error, overcoming the fixed sample size limitation of previous batch auditing methods. Experiments show this test detects violations with sample sizes that are orders of magnitude smaller than existing methods, reducing this number from 50K to a few hundred examples, across diverse realistic mechanisms. Notably, it identifies DP-SGD privacy violations in textitunder one training run, unlike prior methods needing full model training.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 27 of its 50 references that are in the map. First posted on arXiv 8 Sep 2025.

Builds on

27 of 50 references.

References · 27

  1. 2016Deep Learning with Differential Privacycited by 2,728
  2. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 417
  3. 2013The Composition Theorem for Differential Privacycited by 297
  4. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 218
  5. 2020Estimating means of bounded random variables by bettingcited by 169
  6. 2022Reconstructing Training Data with Informed Adversariescited by 148
  7. 2021MMD Aggregated Two-Sample Testcited by 75
  8. 2024Hypothesis testing with e-valuescited by 74
  9. 2021There are natural scores: Full comment on Shafer, "Testing by betting: A strategy for statistical and scientific communication"cited by 53
  10. 2022Connect the Dots: Tighter Discrete Approximations of Privacy Loss Distributionscited by 52
  11. 2020Admissible anytime-valid sequential inference must rely on nonnegative martingalescited by 48
  12. 2022Bounding Training Data Reconstruction in Private (Deep) Learningcited by 46
15 more
  1. 2018Detecting Violations of Differential Privacycited by 39
  2. 2021Nonparametric Two-Sample Testing by Bettingcited by 37
  3. 2023One-shot Empirical Privacy Estimation for Federated Learningcited by 36
  4. 2024Auditing f-Differential Privacy in One Runcited by 30
  5. 2016Understanding the Sparse Vector Technique for Differential Privacycited by 27
  6. 2023Auditing Fairness by Bettingcited by 25
  7. 2020Variational Representations and Neural Network Estimation of R'enyi Divergencescited by 20
  8. 2021Martingale Methods for Sequential Estimation of Convex Functionals and Divergencescited by 18
  9. 2018Property Testing for Differential Privacycited by 16
  10. 2025Instance-Log-Optimality of Portfolio-Based E-Processes and their Sequential Hypothesis Testscited by 12
  11. 2023DP-Auditorium: a Large Scale Library for Auditing Differential Privacycited by 11
  12. 2024An Auditing Test To Detect Behavioral Shift in Language Modelscited by 9
  13. 2021RiLACS: Risk-Limiting Audits via Confidence Sequencescited by 6
  14. 2023Risk-limiting Financial Audits via Weighted Sampling without Replacementcited by 6
  15. 2022Auditing Differential Privacy in High Dimensions with the Kernel Quantum R'enyi Divergencecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Optimal Rates for Differentially Private Hypothesis Testing with E-valuescited by 1
  2. 2026Sequential Auditing for f-Differential Privacycited by 0
  3. 2026Incentive Aware AI Regulations: A Credal Characterisationcited by 0
  4. 2026Sequential Audit Sampling for Finite Populations with Exact and Simulation-based Guaranteecited by 0
  5. 2026Picture the Epsilon: Pursuing Identity-Level Privacy Guarantees for Imagescited by 0