mlsanityData to 14 Oct 2026

Pre-trained Perceptual Features Improve Differentially Private Image Generation

arXiv 2205.12900 ↗First posted 25 May 2022stat.ML cs.CR cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Training even moderately-sized generative models with differentially-private stochastic gradient descent (DP-SGD) is difficult: the required level of noise for reasonable levels of privacy is simply too large. We advocate instead building off a good, relevant representation on an informative public dataset, then learning to model the private data with that representation. In particular, we minimize the maximum mean discrepancy (MMD) between private target data and a generator's distribution, using a kernel based on perceptual features learned from a public dataset. With the MMD, we can simply privatize the data-dependent term once and for all, rather than introducing noise at each step of optimization as in DP-SGD. Our algorithm allows us to generate CIFAR10-level images with ε ≈ 2 which capture distinctive features in the distribution, far surpassing the current state of the art, which mostly focuses on datasets such as MNIST and FashionMNIST at a large ε ≈ 10. Our work introduces simple yet powerful foundations for reducing the gap between private and non-private deep generative models. Our code is available at urlhttps://github.com/ParkLabML/DP-MEPF.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 30 of its 67 references that are in the map. First posted on arXiv 25 May 2022.

Builds on

30 of 67 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2022Generative Adversarial Networkscited by 5,627
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  7. 2016Deep Learning with Differential Privacycited by 2,728
  8. 2018Group Normalizationcited by 1,524
  9. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
  10. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  11. 2015Generative Moment Matching Networkscited by 257
  12. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 236
18 more
  1. 2018Differentially Private Generative Adversarial Networkcited by 235
  2. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 235
  3. 2015Training generative neural networks via Maximum Mean Discrepancy optimizationcited by 151
  4. 2020DP-CGAN: Differentially Private Synthetic Data and Label Generationcited by 118
  5. 2020GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generatorscited by 117
  6. 2020On the Privacy Risks of Algorithmic Fairnesscited by 82
  7. 2020DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generationcited by 78
  8. 2018Differentially Private Fair Learningcited by 76
  9. 2021Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergencecited by 54
  10. 2020Differentially Private and Fair Deep Learning: A Lagrangian Dual Approachcited by 53
  11. 2017Differentially Private Mixture of Generative Neural Networkscited by 39
  12. 2019Differentially Private Generative Adversarial Networks for Time Series, Continuous, and Discrete Open Datacited by 31
  13. 2021PEARL: Data Synthesis via Private Embeddings and Adversarial Reconstruction Learningcited by 23
  14. 2021Hermite Polynomial Features for Private Data Generationcited by 21
  15. 2020Differentially Private and Fair Classification via Calibrated Functional Mechanismcited by 20
  16. 2019Learning Implicit Generative Models by Matching Perceptual Featurescited by 18
  17. 2021Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regressioncited by 17
  18. 2018pMSE Mechanism: Differentially Private Synthetic Data with Maximal Distributional Similaritycited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 7; 2025: 8; 2026: 4.

28 papers.

Sort:

By area:

  1. 2023Differentially Private Synthetic Data via Foundation Model APIs 1: Imagescited by 65
  2. 2023DP-LDMs: Differentially Private Latent Diffusion Modelscited by 29
  3. 2024Differentially Private Fine-Tuning of Diffusion Modelscited by 20
  4. 2023Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generationcited by 15
  5. 2023PrivImage: Differentially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretrainingcited by 13
  6. 2025Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Modelcited by 13
  7. 2025DPImageBench: A Unified Benchmark for Differentially Private Image Synthesiscited by 11
  8. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8
  9. 2025From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesiscited by 7
  10. 2024Clip Body and Tail Separately: High Probability Guarantees for DPSGD with Heavy Tailscited by 5
  11. 2023Differentially Private Gradient Flow based on the Sliced Wasserstein Distancecited by 4
  12. 2024DP-RDM: Adapting Diffusion Models to Private Domains Without Fine-Tuningcited by 4
16 more
  1. 2025Generating Synthetic Data with Formal Privacy Guarantees: State of the Art and the Road Aheadcited by 4
  2. 2025RAPID: Retrieval Augmented Training of Differentially Private Diffusion Modelscited by 3
  3. 2024Efficient Differentially Private Fine-Tuning of Diffusion Modelscited by 2
  4. 2024Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Trainingcited by 2
  5. 2023Differentially Private Kernel Inducing Points using features from ScatterNets (DP-KIP-ScatterNet) for Privacy Preserving Data Distillationcited by 1
  6. 2023PAC Privacy Preserving Diffusion Modelscited by 1
  7. 2024Bayesian Power Steering: An Effective Approach for Domain Adaptation of Diffusion Modelscited by 1
  8. 2025Optimal Differentially Private Sampling of Unbounded Gaussianscited by 1
  9. 2025DP-DocLDM: Differentially Private Document Image Generation using Latent Diffusion Modelscited by 1
  10. 2023DPAF: Image Synthesis via Differentially Private Aggregation in Forward Phasecited by 0
  11. 2023DPGOMI: Differentially Private Data Publishing with Gaussian Optimized Model Inversioncited by 0
  12. 2025Private Training & Data Generation by Clustering Embeddingscited by 0
  13. 2026From Easy to Hard++: Promoting Differentially Private Image Synthesis Through Spatial-Frequency Curriculumcited by 0
  14. 2026Rethinking Benchmarks for Differentially Private Image Classificationcited by 0
  15. 2026DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuningcited by 0
  16. 2026Protecting and Preserving Protest Dynamics for Responsible Analysiscited by 0