mlsanityData to 7 Oct 2026

DP2-VAE: Differentially Private Pre-trained Variational Autoencoders

arXiv 2208.03409 ↗First posted 5 Aug 2022cs.LG cs.CRcited by 13

Not on the map. Why?

Abstract (arXiv)

Modern machine learning systems achieve great success when trained on large datasets. However, these datasets usually contain sensitive information (e.g. medical records, face images), leading to serious privacy concerns. Differentially private generative models (DPGMs) emerge as a solution to circumvent such privacy concerns by generating privatized sensitive data. Similar to other differentially private (DP) learners, the major challenge for DPGM is also how to achieve a subtle balance between utility and privacy. We propose DP2-VAE, a novel training mechanism for variational autoencoders (VAE) with provable DP guarantees and improved utility via pre-training on private data. Under the same DP constraints, DP2-VAE minimizes the perturbation noise during training, and hence improves utility. DP2-VAE is very flexible and easily amenable to many other VAE variants. Theoretically, we study the effect of pretraining on private data. Empirically, we conduct extensive experiments on image datasets to illustrate our superiority over baselines under various privacy budgets and evaluation metrics.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 28 of its 49 references that are in the map. First posted on arXiv 5 Aug 2022.

Builds on

28 of 49 references.

References · 28

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  6. 2016Deep Learning with Differential Privacycited by 2,682
  7. 2018Which Training Methods for GANs do actually Converge?cited by 672
  8. 2017Renyi Differential Privacycited by 643
  9. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  10. 2018Assessing Generative Models via Precision and Recallcited by 401
  11. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  12. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
16 more
  1. 2018Scalable Private Learning with PATEcited by 266
  2. 2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 237
  3. 2018Differentially Private Generative Adversarial Networkcited by 235
  4. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
  5. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  6. 2020DP-CGAN: Differentially Private Synthetic Data and Label Generationcited by 118
  7. 2020GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generatorscited by 117
  8. 2019Generative Models for Effective ML on Private, Decentralized Datasetscited by 85
  9. 2020DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generationcited by 78
  10. 2021Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergencecited by 54
  11. 2019G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminatorscited by 53
  12. 2017Differentially Private Mixture of Generative Neural Networkscited by 39
  13. 2018Differentially Private Data Generative Modelscited by 33
  14. 2021DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregationcited by 31
  15. 2021Differentially Private Normalizing Flows for Privacy-Preserving Density Estimationcited by 10
  16. 2020Differentially Private Variational Autoencoders with Term-wise Gradient Aggregationcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 8; 2024: 0; 2025: 4.

13 papers.

Sort:

By area:

  1. 2023Differentially Private Diffusion Models Generate Useful Synthetic Imagescited by 85
  2. 2023Why Is Public Pretraining Necessary for Private Model Training?cited by 36
  3. 2023DP-LDMs: Differentially Private Latent Diffusion Modelscited by 28
  4. 2022DPD-fVAE: Synthetic Data Generation Using Federated Variational Autoencoders With Differentially-Private Decodercited by 14
  5. 2023PrivImage: Differentially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretrainingcited by 12
  6. 2025From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesiscited by 6
  7. 2023Quantifying Sample Anonymity in Score-Based Generative Models with Adversarial Fingerprintingcited by 4
  8. 2023A Unified View of Differentially Private Deep Generative Modelingcited by 4
  9. 2025DELTA: Variational Disentangled Learning for Privacy-Preserving Data Reprogrammingcited by 3
  10. 2025RAPID: Retrieval Augmented Training of Differentially Private Diffusion Modelscited by 2
  11. 2025SoK: Can Synthetic Images Replace Real Data? A Survey of Utility and Privacy of Synthetic Image Generationcited by 1
  12. 2023DPAF: Image Synthesis via Differentially Private Aggregation in Forward Phasecited by 0
1 more
  1. 2023PAC Privacy Preserving Diffusion Modelscited by 0