Privacy Risk for anisotropic Langevin dynamics using relative entropy bounds
Not on the map. Why?
Abstract (arXiv)
The privacy preserving properties of Langevin dynamics with additive isotropic noise have been extensively studied. However, the isotropic noise assumption is very restrictive: (a) when adding noise to existing learning algorithms to preserve privacy and maintain the best possible accuracy one should take into account the relative magnitude of the outputs and their correlations; (b) popular algorithms such as stochastic gradient descent (and their continuous time limits) appear to possess anisotropic covariance properties. To study the privacy risks for the anisotropic noise case, one requires general results on the relative entropy between the laws of two Stochastic Differential Equations with different drifts and diffusion coefficients. Our main contribution is to establish such a bound using stability estimates for solutions to the Fokker-Planck equations via functional inequalities. With additional assumptions, the relative entropy bound implies an (ε,δ)-differential privacy bound or translates to bounds on the membership inference attack success and we show how anisotropic noise can lead to better privacy-accuracy trade-offs. Finally, the benefits of anisotropic noise are illustrated using numerical results in quadratic loss and neural network setups.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 16 of its 53 references that are in the map. First posted on arXiv 1 Feb 2023.
Builds on
16 of 53 references.
References · 16
- 2016Deep Learning with Differential Privacycited by 2,682
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
- 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
- 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
- 2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 216
- 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
- 2015Stochastic modified equations and adaptive stochastic gradient algorithmscited by 141
- 2017Stochastic gradient descent performs variational inference, converges to limit cycles for deep networkscited by 108
- 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 71
- 2018Sampling Can Be Faster Than Optimizationcited by 68
- 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 60
- 2020Faster Differentially Private Samplers via R'enyi Divergence Analysis of Discretized Langevin MCMCcited by 27
4 more
- 2022Optimal Membership Inference Bounds for Adaptive Composition of Sampled Gaussian Mechanismscited by 19
- 2018MVG Mechanism: Differential Privacy under Matrix-Valued Querycited by 6
- 2021On Large Batch Training and Sharp Minima: A Fokker-Planck Perspectivecited by 5
- 2021Improved Matrix Gaussian Mechanism for Differential Privacycited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.