# Picture the Epsilon: Pursuing Identity-Level Privacy Guarantees for Images

arXiv: https://arxiv.org/abs/2608.17147 · HTML: https://mlsanity.com/paper/2608.17147/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 17 Aug 2026.

## Builds on (most cited first, 22 of 22)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [The Composition Theorem for Differential Privacy](https://mlsanity.com/paper/1311.0776/) (2013)
- [Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising](https://mlsanity.com/paper/1805.06530/) (2018)
- [Auditing Differentially Private Machine Learning: How Private is Private SGD?](https://mlsanity.com/paper/2006.07709/) (2020)
- [Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning](https://mlsanity.com/paper/2101.04535/) (2021)
- [The Discrete Gaussian for Differential Privacy](https://mlsanity.com/paper/2004.00010/) (2020)
- [Privacy Auditing with One (1) Training Run](https://mlsanity.com/paper/2305.08846/) (2023)
- [Membership Inference of Diffusion Models](https://mlsanity.com/paper/2301.09956/) (2023)
- [Detecting Violations of Differential Privacy](https://mlsanity.com/paper/1805.10277/) (2018)
- [IdentityDP: Differential Private Identification Protection for Face Images](https://mlsanity.com/paper/2103.01745/) (2021)
- [DP-Image: Differential Privacy for Image Data in Feature Space](https://mlsanity.com/paper/2103.07073/) (2021)
- [Protecting Global Properties of Datasets with Distribution Privacy Mechanisms](https://mlsanity.com/paper/2207.08367/) (2022)
- [DP-Auditorium: a Large Scale Library for Auditing Differential Privacy](https://mlsanity.com/paper/2307.05608/) (2023)
- [General Framework to Evaluate Unlinkability in Biometric Template Protection Systems](https://mlsanity.com/paper/2311.04633/) (2023)
- [Privacy Audit as Bits Transmission: (Im)possibilities for Audit by One Run](https://mlsanity.com/paper/2501.17750/) (2025)
- [Local Obfuscation Mechanisms for Hiding Probability Distributions](https://mlsanity.com/paper/1812.00939/) (2018)
- [Sequentially Auditing Differential Privacy](https://mlsanity.com/paper/2509.07055/) (2025)
- [Auditing Differential Privacy in High Dimensions with the Kernel Quantum R'enyi Divergence](https://mlsanity.com/paper/2205.13941/) (2022)

## Abstract (arXiv)

Several methods for auditing privacy in embedding spaces report a number called "epsilon." The common name is misleading: one number may be a heuristic score, another may come from a valid population inequality but ignore sampling uncertainty, and a third may be a confidence bound. This paper asks when such a number is evidence about differential privacy. We study four approaches based on Gaussian calibration, marginal kernel-density ratios, maximum mean discrepancy (MMD), and classifier hypothesis tests. The first two are modeling diagnostics. The MMD and classifier approaches use valid population lower bounds, but only the classifier approach is given a separate test set and a finite-sample confidence calculation. To see how these distinctions matter, we build a synthetic benchmark in which the true privacy value is known. It includes pure-DP Laplace mechanisms, approximate-DP Gaussian mechanisms, an exact privacy null, two identity geometries, and three sample sizes. At the null, the Gaussian and kernel-density diagnostics remain large. A direct conversion of an empirical ROC curve often returns infinity even though the AUC is near chance and no threshold separates the samples perfectly. The MMD value increases as the distributions become easier to distinguish, but the sample estimate is small relative to the known reference and is not a lower confidence bound. By contrast, a classifier chosen on development data and evaluated on untouched test data gives simultaneous lower confidence bounds under the stated iid model. We also apply the methods to FaceFusion and InstantID. That case study shows how the methods behave on face data but does not calibrate them because the generators have no known privacy value. The main lesson is that an epsilon-like number is meaningful only together with its assumptions and its finite-sample interpretation.