# Differentially Private Statistical Inference through β-Divergence One Posterior Sampling

arXiv: https://arxiv.org/abs/2307.05194 · HTML: https://mlsanity.com/paper/2307.05194/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 87 references that are in the map.
- First posted on arXiv 11 Jul 2023.

## Builds on (most cited first, 30 of 43)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Unlocking High-Accuracy Differentially Private Image Classification through Scale](https://mlsanity.com/paper/2204.13650/) (2022)
- [Auditing Differentially Private Machine Learning: How Private is Private SGD?](https://mlsanity.com/paper/2006.07709/) (2020)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning](https://mlsanity.com/paper/2101.04535/) (2021)
- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [Synthetic Data -- Anonymisation Groundhog Day](https://mlsanity.com/paper/2011.07018/) (2020)
- [A General Approach to Adding Differential Privacy to Iterative Training Procedures](https://mlsanity.com/paper/1812.06210/) (2018)
- [Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo](https://mlsanity.com/paper/1502.07645/) (2015)
- [Diffprivlib: The IBM Differential Privacy Library](https://mlsanity.com/paper/1907.02444/) (2019)
- [Robust Generalised Bayesian Inference for Intractable Likelihoods](https://mlsanity.com/paper/2104.07359/) (2021)
- [On the Theory and Practice of Privacy-Preserving Bayesian Data Analysis](https://mlsanity.com/paper/1603.07294/) (2016)
- [Faster Differentially Private Samplers via R'enyi Divergence Analysis of Discretized Langevin MCMC](https://mlsanity.com/paper/2010.14658/) (2020)
- [Differentially Private Bayesian Linear Regression](https://mlsanity.com/paper/1910.13153/) (2019)
- [Doubly Robust Bayesian Inference for Non-Stationary Streaming Data with β-Divergences](https://mlsanity.com/paper/1806.02261/) (2018)
- [On the Geometric Ergodicity of Hamiltonian Monte Carlo](https://mlsanity.com/paper/1601.08057/) (2016)
- [Bayesian Differential Privacy through Posterior Sampling](https://mlsanity.com/paper/1306.1066/) (2013)
- [Differentially Private Bayesian Inference for Exponential Families](https://mlsanity.com/paper/1809.02188/) (2018)
- [Differentially Private Markov Chain Monte Carlo](https://mlsanity.com/paper/1901.10275/) (2019)
- [Privacy-preserving parametric inference: a case for robust statistics](https://mlsanity.com/paper/1911.10167/) (2019)
- [Differentially Private Variational Inference for Non-conjugate Models](https://mlsanity.com/paper/1610.08749/) (2016)
- [R'enyi Differential Privacy Mechanisms for Posterior Sampling](https://mlsanity.com/paper/1710.00892/) (2017)
- [On Connecting Stochastic Gradient MCMC and Differential Privacy](https://mlsanity.com/paper/1712.09097/) (2017)
- [Differentially Private Bayesian Inference for Generalized Linear Models](https://mlsanity.com/paper/2011.00467/) (2020)
- [Robust and Scalable Bayesian Online Changepoint Detection](https://mlsanity.com/paper/2302.04759/) (2023)
- [On robustness and local differential privacy](https://mlsanity.com/paper/2201.00751/) (2022)
- [Objective Bayesian inference with proper scoring rules](https://mlsanity.com/paper/1711.10819/) (2017)
- [An Efficient Minibatch Acceptance Test for Metropolis-Hastings](https://mlsanity.com/paper/1610.06848/) (2016)

## Built on by (most cited first, 1 of 1)

- [On Inherent Privacy of Posterior Sampling: A Unified R'enyi-Divergence Framework](https://mlsanity.com/paper/2610.19573/) (2026)

## Abstract (arXiv)

Differential privacy guarantees allow the results of a statistical analysis involving sensitive data to be released without compromising the privacy of any individual taking part. Achieving such guarantees generally requires the injection of noise, either directly into parameter estimates or into the estimation process. Instead of artificially introducing perturbations, sampling from Bayesian posterior distributions has been shown to be a special case of the exponential mechanism, producing consistent, and efficient private estimates without altering the data generative process. The application of current approaches has, however, been limited by their strong bounding assumptions which do not hold for basic models, such as simple linear regressors. To ameliorate this, we propose βD-Bayes, a posterior sampling scheme from a generalised posterior targeting the minimisation of the β-divergence between the model and the data generating process. This provides private estimation that is generally applicable without requiring changes to the underlying model and consistently learns the data generating parameter. We show that βD-Bayes produces more precise inference estimation for the same privacy guarantees, and further facilitates differentially private estimation via posterior sampling for complex classifiers and continuous regression models such as neural networks for the first time.