# Statistic Selection and MCMC for Differentially Private Bayesian Estimation

arXiv: https://arxiv.org/abs/2203.13377 · HTML: https://mlsanity.com/paper/2203.13377/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 13 of 13)

- [Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds](https://mlsanity.com/paper/1605.02065/) (2016)
- [Differentially Private Learning with Adaptive Clipping](https://mlsanity.com/paper/1905.03871/) (2019)
- [AdaCliP: Adaptive Clipping for Private SGD](https://mlsanity.com/paper/1908.07643/) (2019)
- [Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo](https://mlsanity.com/paper/1502.07645/) (2015)
- [On the Theory and Practice of Privacy-Preserving Bayesian Data Analysis](https://mlsanity.com/paper/1603.07294/) (2016)
- [Differentially Private Bayesian Inference for Exponential Families](https://mlsanity.com/paper/1809.02188/) (2018)
- [Differentially Private Markov Chain Monte Carlo](https://mlsanity.com/paper/1901.10275/) (2019)
- [Privacy-preserving parametric inference: a case for robust statistics](https://mlsanity.com/paper/1911.10167/) (2019)
- [On Connecting Stochastic Gradient MCMC and Differential Privacy](https://mlsanity.com/paper/1712.09097/) (2017)
- [Differentially Private Exponential Random Graphs](https://mlsanity.com/paper/1409.4696/) (2014)
- [Metropolis-Hastings with Averaged Acceptance Ratios](https://mlsanity.com/paper/2101.01253/) (2020)
- [Differentially Private Hamiltonian Monte Carlo](https://mlsanity.com/paper/2106.09376/) (2021)
- [Scalable Monte Carlo inference for state-space models](https://mlsanity.com/paper/1809.02527/) (2018)

## Built on by (most cited first, 3 of 3)

- [Differentially Private Online Bayesian Estimation With Adaptive Truncation](https://mlsanity.com/paper/2301.08202/) (2023)
- [Differentially Private Distributed Bayesian Linear Regression with MCMC](https://mlsanity.com/paper/2301.13778/) (2023)
- [Bayesian Frequency Estimation Under Local Differential Privacy With an Adaptive Randomized Response Mechanism](https://mlsanity.com/paper/2405.07020/) (2024)

## Abstract (arXiv)

This paper concerns differentially private Bayesian estimation of the parameters of a population distribution, when a statistic of a sample from that population is shared in noise to provide differential privacy. This work mainly addresses two problems: (1) What statistic of the sample should be shared privately? For the first question, i.e., the one about statistic selection, we promote using the Fisher information. We find out that, the statistic that is most informative in a non-privacy setting may not be the optimal choice under the privacy restrictions. We provide several examples to support that point. We consider several types of data sharing settings and propose several Monte Carlo-based numerical estimation methods for calculating the Fisher information for those settings. The second question concerns inference: (2) Based on the shared statistics, how could we perform effective Bayesian inference? We propose several Markov chain Monte Carlo (MCMC) algorithms for sampling from the posterior distribution of the parameter given the noisy statistic. The proposed MCMC algorithms can be preferred over one another depending on the problem. For example, when the shared statistics is additive and added Gaussian noise, a simple Metropolis-Hasting algorithm that utilizes the central limit theorem is a decent choice. We propose more advanced MCMC algorithms for several other cases of practical relevance. Our numerical examples involve comparing several candidate statistics to be shared privately. For each statistic, we perform Bayesian estimation based on the posterior distribution conditional on the privatized version of that statistic. We demonstrate that, the relative performance of a statistic, in terms of the mean squared error of the Bayesian estimator based on the corresponding privatized statistic, is adequately predicted by the Fisher information of the privatized statistic.