Auditing Differential Privacy in the Black-Box Setting
Not on the map. Why?
Abstract (arXiv)
This paper introduces a novel theoretical framework for auditing differential privacy (DP) in a black-box setting. Leveraging the concept of f-differential privacy, we explicitly define type I and type II errors and propose an auditing mechanism based on conformal inference. Our approach robustly controls the type I error rate under minimal assumptions. Furthermore, we establish a fundamental impossibility result, demonstrating the inherent difficulty of simultaneously controlling both type I and type II errors without additional assumptions. Nevertheless, under a monotone likelihood ratio (MLR) assumption, our auditing mechanism effectively controls both errors. We also extend our method to construct valid confidence bands for the trade-off function in the finite-sample regime.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 20 references that are in the map. First posted on arXiv 15 Mar 2025.
Builds on
11 of 20 references.
References · 11
- 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
- 2014RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Responsecited by 272
- 2017Collecting Telemetry Data Privatelycited by 149
- 2023Privacy Auditing with One (1) Training Runcited by 123
- 2019Deep Learning with Gaussian Differential Privacycited by 114
- 2018Differentially Private Contextual Linear Banditscited by 82
- 2024Auditing f-Differential Privacy in One Runcited by 30
- 2022Differentially Private Stochastic Linear Bandits: (Almost) for Freecited by 19
- 2023A Randomized Approach for Tight Privacy Accountingcited by 11
- 2024Auditing Differential Privacy Guarantees Using Density Estimationcited by 6
- 2024Privacy Preserving Adaptive Experiment Designcited by 1
Built on by
Not cited yet.