# On the Theory and Practice of Privacy-Preserving Bayesian Data Analysis

arXiv: https://arxiv.org/abs/1603.07294 · HTML: https://mlsanity.com/paper/1603.07294/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2016.

## Built on by (most cited first, 30 of 35)

- [Differential privacy and robust statistics in high dimensions](https://mlsanity.com/paper/2111.06578/) (2021)
- [DP-PCA: Statistically Optimal and Differentially Private PCA](https://mlsanity.com/paper/2205.13709/) (2022)
- [Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimation](https://mlsanity.com/paper/2402.13531/) (2024)
- [Incentives in Private Collaborative Machine Learning](https://mlsanity.com/paper/2404.01676/) (2024)
- [Near Optimal Private and Robust Linear Regression](https://mlsanity.com/paper/2301.13273/) (2023)
- [Non-parametric Differentially Private Confidence Intervals for the Median](https://mlsanity.com/paper/2106.10333/) (2021)
- [Insufficient Statistics Perturbation: Stable Estimators for Private Least Squares](https://mlsanity.com/paper/2404.15409/) (2024)
- [A Meta-learning Framework for Tuning Parameters of Protection Mechanisms in Trustworthy Federated Learning](https://mlsanity.com/paper/2305.18400/) (2023)
- [Protect Your Score: Contact Tracing With Differential Privacy Guarantees](https://mlsanity.com/paper/2312.11581/) (2023)
- [Privacy Amplification Via Bernoulli Sampling](https://mlsanity.com/paper/2105.10594/) (2021)
- [Mitigating Statistical Bias within Differentially Private Synthetic Data](https://mlsanity.com/paper/2108.10934/) (2021)
- [Private Regression via Data-Dependent Sufficient Statistic Perturbation](https://mlsanity.com/paper/2405.15002/) (2024)
- [Statistic Selection and MCMC for Differentially Private Bayesian Estimation](https://mlsanity.com/paper/2203.13377/) (2022)
- [Multi-Task Differential Privacy Under Distribution Skew](https://mlsanity.com/paper/2302.07975/) (2023)
- [DP-Fast MH: Private, Fast, and Accurate Metropolis-Hastings for Large-Scale Bayesian Inference](https://mlsanity.com/paper/2303.06171/) (2023)
- [Private Matrix Factorization with Public Item Features](https://mlsanity.com/paper/2309.11516/) (2023)
- [Dirichlet Mechanism for Differentially Private KL Divergence Minimization](https://mlsanity.com/paper/2110.01984/) (2021)
- [Can Stochastic Gradient Langevin Dynamics Provide Differential Privacy for Deep Learning?](https://mlsanity.com/paper/2110.05057/) (2021)
- [Differentially private partitioned variational inference](https://mlsanity.com/paper/2209.11595/) (2022)
- [Purifying Approximate Differential Privacy with Randomized Post-processing](https://mlsanity.com/paper/2503.21071/) (2025)
- [Locally Differentially Private Bayesian Inference](https://mlsanity.com/paper/2110.14426/) (2021)
- [Differentially private stochastic expectation propagation (DP-SEP)](https://mlsanity.com/paper/2111.13219/) (2021)
- [Differentially Private Online Bayesian Estimation With Adaptive Truncation](https://mlsanity.com/paper/2301.08202/) (2023)
- [Differentially Private Distributed Bayesian Linear Regression with MCMC](https://mlsanity.com/paper/2301.13778/) (2023)
- [Differentially Private Statistical Inference through β-Divergence One Posterior Sampling](https://mlsanity.com/paper/2307.05194/) (2023)
- [Simulation-based Bayesian Inference from Privacy Protected Data](https://mlsanity.com/paper/2310.12781/) (2023)
- [Bridging Privacy and Robustness for Trustworthy Machine Learning](https://mlsanity.com/paper/2403.16591/) (2024)
- [Bayesian Frequency Estimation Under Local Differential Privacy With an Adaptive Randomized Response Mechanism](https://mlsanity.com/paper/2405.07020/) (2024)
- [Distributed Differentially Private Data Analytics via Secure Sketching](https://mlsanity.com/paper/2412.00497/) (2024)
- [Near-Optimal Private Linear Regression via Iterative Hessian Mixing](https://mlsanity.com/paper/2601.07545/) (2026)

## Abstract (arXiv)

Bayesian inference has great promise for the privacy-preserving analysis of sensitive data, as posterior sampling automatically preserves differential privacy, an algorithmic notion of data privacy, under certain conditions (Dimitrakakis et al., 2014; Wang et al., 2015). While this one posterior sample (OPS) approach elegantly provides privacy "for free," it is data inefficient in the sense of asymptotic relative efficiency (ARE). We show that a simple alternative based on the Laplace mechanism, the workhorse of differential privacy, is as asymptotically efficient as non-private posterior inference, under general assumptions. This technique also has practical advantages including efficient use of the privacy budget for MCMC. We demonstrate the practicality of our approach on a time-series analysis of sensitive military records from the Afghanistan and Iraq wars disclosed by the Wikileaks organization.