mlsanityData to 14 Oct 2026

Detecting Violations of Differential Privacy

arXiv 1805.10277 ↗First posted 25 May 2018cs.CRcited by 39

Not on the map. Why?

Abstract (arXiv)

The widespread acceptance of differential privacy has led to the publication of many sophisticated algorithms for protecting privacy. However, due to the subtle nature of this privacy definition, many such algorithms have bugs that make them violate their claimed privacy. In this paper, we consider the problem of producing counterexamples for such incorrect algorithms. The counterexamples are designed to be short and human-understandable so that the counterexample generator can be used in the development process -- a developer could quickly explore variations of an algorithm and investigate where they break down. Our approach is statistical in nature. It runs a candidate algorithm many times and uses statistical tests to try to detect violations of differential privacy. An evaluation on a variety of incorrect published algorithms validates the usefulness of our approach: it correctly rejects incorrect algorithms and provides counterexamples for them within a few seconds.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 5; 2023: 7; 2024: 6; 2025: 9; 2026: 7.

39 papers.

Sort:

By area:

  1. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 199
  2. 2023Privacy Auditing with One (1) Training Runcited by 123
  3. 2022Synthetic Data -- what, why and how?cited by 67
  4. 2023One-shot Empirical Privacy Estimation for Federated Learningcited by 36
  5. 2022Bayesian Estimation of Differential Privacycited by 33
  6. 2024Auditing f-Differential Privacy in One Runcited by 30
  7. 2022A General Framework for Auditing Differentially Private Machine Learningcited by 25
  8. 2023Unleashing the Power of Randomization in Auditing Differentially Private MLcited by 20
  9. 2021On a Utilitarian Approach to Privacy Preserving Text Generationcited by 17
  10. 2025Differentially Private Steering for Large Language Model Alignmentcited by 15
  11. 2024The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGDcited by 11
  12. 2022How reparametrization trick broke differentially-private text representation learningcited by 10
27 more
  1. 2023Quantum Pufferfish Privacy: A Flexible Privacy Framework for Quantum Systemscited by 9
  2. 2025How Well Can Differential Privacy Be Audited in One Run?cited by 9
  3. 2024To Shuffle or not to Shuffle: Auditing DP-SGD with Shufflingcited by 6
  4. 2025Optimizing Canaries for Privacy Auditing with Metagradient Descentcited by 6
  5. 2024Revisiting Differentially Private Hyper-parameter Tuningcited by 5
  6. 2025The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditingcited by 5
  7. 2025Sequentially Auditing Differential Privacycited by 5
  8. 2025Enhancing One-run Privacy Auditing with Quantile Regression-Based Membership Inferencecited by 4
  9. 2025ContextLeak: Auditing Leakage in Private In-Context Learning Methodscited by 4
  10. 2022Auditing Differential Privacy in High Dimensions with the Kernel Quantum R'enyi Divergencecited by 3
  11. 2023Probing the Transition to Dataset-Level Privacy in ML Models Using an Output-Specific and Data-Resolved Privacy Profilecited by 3
  12. 2023Detecting Violations of Differential Privacy for Quantum Algorithmscited by 3
  13. 2024QueryCheetah: Fast Automated Discovery of Attribute Inference Attacks Against Query-Based Systemscited by 3
  14. 2021Can Stochastic Gradient Langevin Dynamics Provide Differential Privacy for Deep Learning?cited by 2
  15. 2023Epsilon*: Privacy Metric for Machine Learning Modelscited by 2
  16. 2025A Unified Framework for Adversary-Aware Differential Privacy Boundscited by 2
  17. 2026Understanding Disclosure Risk in Differential Privacy with Applications to Noise Calibration and Auditing (Extended Version)cited by 2
  18. 2021Statistical Privacy Guarantees of Machine Learning Preprocessing Techniquescited by 1
  19. 2021Learning Differentially Private Mechanismscited by 0
  20. 2024Adversarial Sample-Based Approach for Tighter Privacy Auditing in Final Model-Only Scenarioscited by 0
  21. 2025Observational Auditing of Label Privacycited by 0
  22. 2026DPrivBench: Benchmarking LLMs' Reasoning for Differential Privacycited by 0
  23. 2026When Are Trade-Off Functions Testable from Finite Samples?cited by 0
  24. 2026Optimal Guarantees for Auditing R'enyi Differentially Private Machine Learningcited by 0
  25. 2026Detectability in Diversity: Improved Canary Crafting for Privacy Auditing in One Runcited by 0
  26. 2026Near-optimal node-private community estimation in polynomial-timecited by 0
  27. 2026Picture the Epsilon: Pursuing Identity-Level Privacy Guarantees for Imagescited by 0