# Differentially Private Variational Inference for Non-conjugate Models

arXiv: https://arxiv.org/abs/1610.08749 · HTML: https://mlsanity.com/paper/1610.08749/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2016.

## Built on by (most cited first, 16 of 16)

- [Survey: Leakage and Privacy at Inference Time](https://mlsanity.com/paper/2107.01614/) (2021)
- [Partitioned Variational Inference: A Framework for Probabilistic Federated Learning](https://mlsanity.com/paper/2202.12275/) (2022)
- [Noise-Aware Statistical Inference with Differentially Private Synthetic Data](https://mlsanity.com/paper/2205.14485/) (2022)
- [Federated Variational Inference Methods for Structured Latent Variable Models](https://mlsanity.com/paper/2302.03314/) (2023)
- [Gaussian Processes with Differential Privacy](https://mlsanity.com/paper/2106.00474/) (2021)
- [D3p -- A Python Package for Differentially-Private Probabilistic Programming](https://mlsanity.com/paper/2103.11648/) (2021)
- [Differentially private partitioned variational inference](https://mlsanity.com/paper/2209.11595/) (2022)
- [DPVIm: Differentially Private Variational Inference Improved](https://mlsanity.com/paper/2210.15961/) (2022)
- [Differentially Private Normalizing Flows for Density Estimation, Data Synthesis, and Variational Inference with Application to Electronic Health Records](https://mlsanity.com/paper/2302.05787/) (2023)
- [A Bias-Variance Decomposition for Ensembles over Multiple Synthetic Datasets](https://mlsanity.com/paper/2402.03985/) (2024)
- [Locally Differentially Private Bayesian Inference](https://mlsanity.com/paper/2110.14426/) (2021)
- [Differentially private stochastic expectation propagation (DP-SEP)](https://mlsanity.com/paper/2111.13219/) (2021)
- [Differentially Private Statistical Inference through β-Divergence One Posterior Sampling](https://mlsanity.com/paper/2307.05194/) (2023)
- [Collaborative Learning From Distributed Data With Differentially Private Synthetic Twin Data](https://mlsanity.com/paper/2308.04755/) (2023)
- [Computation-Utility-Privacy Tradeoffs in Bayesian Estimation](https://mlsanity.com/paper/2603.18254/) (2026)
- [Noise-Aware Differentially Private Variational Inference](https://mlsanity.com/paper/2410.19371/) (2024)

## Abstract (arXiv)

Many machine learning applications are based on data collected from people, such as their tastes and behaviour as well as biological traits and genetic data. Regardless of how important the application might be, one has to make sure individuals' identities or the privacy of the data are not compromised in the analysis. Differential privacy constitutes a powerful framework that prevents breaching of data subject privacy from the output of a computation. Differentially private versions of many important Bayesian inference methods have been proposed, but there is a lack of an efficient unified approach applicable to arbitrary models. In this contribution, we propose a differentially private variational inference method with a very wide applicability. It is built on top of doubly stochastic variational inference, a recent advance which provides a variational solution to a large class of models. We add differential privacy into doubly stochastic variational inference by clipping and perturbing the gradients. The algorithm is made more efficient through privacy amplification from subsampling. We demonstrate the method can reach an accuracy close to non-private level under reasonably strong privacy guarantees, clearly improving over previous sampling-based alternatives especially in the strong privacy regime.