# Differentially Private Mixture of Generative Neural Networks

arXiv: https://arxiv.org/abs/1709.04514 · HTML: https://mlsanity.com/paper/1709.04514/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2017.

## Built on by (most cited first, 30 of 39)

- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [Differentially Private Diffusion Models](https://mlsanity.com/paper/2210.09929/) (2022)
- [Synthetic Data -- what, why and how?](https://mlsanity.com/paper/2205.03257/) (2022)
- [Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence](https://mlsanity.com/paper/2111.01177/) (2021)
- [Robin Hood and Matthew Effects: Differential Privacy Has Disparate Impact on Synthetic Data](https://mlsanity.com/paper/2109.11429/) (2021)
- [SoK: Privacy-Preserving Data Synthesis](https://mlsanity.com/paper/2307.02106/) (2023)
- [DP-LDMs: Differentially Private Latent Diffusion Models](https://mlsanity.com/paper/2305.15759/) (2023)
- [Pre-trained Perceptual Features Improve Differentially Private Image Generation](https://mlsanity.com/paper/2205.12900/) (2022)
- [Hermite Polynomial Features for Private Data Generation](https://mlsanity.com/paper/2106.05042/) (2021)
- [Wide Network Learning with Differential Privacy](https://mlsanity.com/paper/2103.01294/) (2021)
- [Generative Adversarial Networks: A Survey Towards Private and Secure Applications](https://mlsanity.com/paper/2106.03785/) (2021)
- [SGDE: Secure Generative Data Exchange for Cross-Silo Federated Learning](https://mlsanity.com/paper/2109.12062/) (2021)
- [DPD-fVAE: Synthetic Data Generation Using Federated Variational Autoencoders With Differentially-Private Decoder](https://mlsanity.com/paper/2211.11591/) (2022)
- [Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning](https://mlsanity.com/paper/2303.04274/) (2023)
- [Differentially Private Imaging via Latent Space Manipulation](https://mlsanity.com/paper/2103.05472/) (2021)
- [DP2-VAE: Differentially Private Pre-trained Variational Autoencoders](https://mlsanity.com/paper/2208.03409/) (2022)
- [The Elusive Pursuit of Reproducing PATE-GAN: Benchmarking, Auditing, Debugging](https://mlsanity.com/paper/2406.13985/) (2024)
- [dpart: Differentially Private Autoregressive Tabular, a General Framework for Synthetic Data Generation](https://mlsanity.com/paper/2207.05810/) (2022)
- [Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey](https://mlsanity.com/paper/2309.16398/) (2023)
- [The DCR Delusion: Measuring the Privacy Risk of Synthetic Data](https://mlsanity.com/paper/2505.01524/) (2025)
- [On Utility and Privacy in Synthetic Genomic Data](https://mlsanity.com/paper/2102.03314/) (2021)
- [Generating tabular datasets under differential privacy](https://mlsanity.com/paper/2308.14784/) (2023)
- [Tabular Data Synthesis with Differential Privacy: A Survey](https://mlsanity.com/paper/2411.03351/) (2024)
- [Differentially Private Transferrable Deep Learning with Membership-Mappings](https://mlsanity.com/paper/2105.04615/) (2021)
- [Photonic Differential Privacy with Direct Feedback Alignment](https://mlsanity.com/paper/2106.03645/) (2021)
- [A Systematic Review of Federated Generative Models](https://mlsanity.com/paper/2405.16682/) (2024)
- [The Importance of Being Discrete: Measuring the Impact of Discretization in End-to-End Differentially Private Synthetic Data](https://mlsanity.com/paper/2504.06923/) (2025)
- [Mitigating Statistical Bias within Differentially Private Synthetic Data](https://mlsanity.com/paper/2108.10934/) (2021)
- [A Unified View of Differentially Private Deep Generative Modeling](https://mlsanity.com/paper/2309.15696/) (2023)
- [TraVaG: Differentially Private Trace Variant Generation Using GANs](https://mlsanity.com/paper/2303.16704/) (2023)

## Abstract (arXiv)

Generative models are used in a wide range of applications building on large amounts of contextually rich information. Due to possible privacy violations of the individuals whose data is used to train these models, however, publishing or sharing generative models is not always viable. In this paper, we present a novel technique for privately releasing generative models and entire high-dimensional datasets produced by these models. We model the generator distribution of the training data with a mixture of k generative neural networks. These are trained together and collectively learn the generator distribution of a dataset. Data is divided into k clusters, using a novel differentially private kernel k-means, then each cluster is given to separate generative neural networks, such as Restricted Boltzmann Machines or Variational Autoencoders, which are trained only on their own cluster using differentially private gradient descent. We evaluate our approach using the MNIST dataset, as well as call detail records and transit datasets, showing that it produces realistic synthetic samples, which can also be used to accurately compute arbitrary number of counting queries.