# Pre-trained Perceptual Features Improve Differentially Private Image Generation

arXiv: https://arxiv.org/abs/2205.12900 · HTML: https://mlsanity.com/paper/2205.12900/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 67 references that are in the map.
- First posted on arXiv 25 May 2022.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Differentially Private Generative Adversarial Network](https://mlsanity.com/paper/1802.06739/) (2018)
- [Differentially Private Learning Needs Better Features (or Much More Data)](https://mlsanity.com/paper/2011.11660/) (2020)
- [Training generative neural networks via Maximum Mean Discrepancy optimization](https://mlsanity.com/paper/1505.03906/) (2015)
- [DP-CGAN: Differentially Private Synthetic Data and Label Generation](https://mlsanity.com/paper/2001.09700/) (2020)
- [GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators](https://mlsanity.com/paper/2006.08265/) (2020)
- [On the Privacy Risks of Algorithmic Fairness](https://mlsanity.com/paper/2011.03731/) (2020)
- [DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generation](https://mlsanity.com/paper/2002.11603/) (2020)
- [Differentially Private Fair Learning](https://mlsanity.com/paper/1812.02696/) (2018)
- [Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence](https://mlsanity.com/paper/2111.01177/) (2021)
- [Differentially Private and Fair Deep Learning: A Lagrangian Dual Approach](https://mlsanity.com/paper/2009.12562/) (2020)
- [Differentially Private Mixture of Generative Neural Networks](https://mlsanity.com/paper/1709.04514/) (2017)
- [Differentially Private Generative Adversarial Networks for Time Series, Continuous, and Discrete Open Data](https://mlsanity.com/paper/1901.02477/) (2019)
- [PEARL: Data Synthesis via Private Embeddings and Adversarial Reconstruction Learning](https://mlsanity.com/paper/2106.04590/) (2021)
- [Hermite Polynomial Features for Private Data Generation](https://mlsanity.com/paper/2106.05042/) (2021)
- [Differentially Private and Fair Classification via Calibrated Functional Mechanism](https://mlsanity.com/paper/2001.04958/) (2020)
- [Learning Implicit Generative Models by Matching Perceptual Features](https://mlsanity.com/paper/1904.02762/) (2019)
- [Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression](https://mlsanity.com/paper/2112.04470/) (2021)
- [pMSE Mechanism: Differentially Private Synthetic Data with Maximal Distributional Similarity](https://mlsanity.com/paper/1805.09392/) (2018)

## Built on by (most cited first, 28 of 28)

- [Differentially Private Synthetic Data via Foundation Model APIs 1: Images](https://mlsanity.com/paper/2305.15560/) (2023)
- [DP-LDMs: Differentially Private Latent Diffusion Models](https://mlsanity.com/paper/2305.15759/) (2023)
- [Differentially Private Fine-Tuning of Diffusion Models](https://mlsanity.com/paper/2406.01355/) (2024)
- [Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generation](https://mlsanity.com/paper/2303.01687/) (2023)
- [PrivImage: Differentially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretraining](https://mlsanity.com/paper/2311.12850/) (2023)
- [Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model](https://mlsanity.com/paper/2502.05505/) (2025)
- [DPImageBench: A Unified Benchmark for Differentially Private Image Synthesis](https://mlsanity.com/paper/2503.14681/) (2025)
- [A Survey on Responsible Generative AI: What to Generate and What Not](https://mlsanity.com/paper/2404.05783/) (2024)
- [From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesis](https://mlsanity.com/paper/2504.01395/) (2025)
- [Clip Body and Tail Separately: High Probability Guarantees for DPSGD with Heavy Tails](https://mlsanity.com/paper/2405.17529/) (2024)
- [Differentially Private Gradient Flow based on the Sliced Wasserstein Distance](https://mlsanity.com/paper/2312.08227/) (2023)
- [DP-RDM: Adapting Diffusion Models to Private Domains Without Fine-Tuning](https://mlsanity.com/paper/2403.14421/) (2024)
- [Generating Synthetic Data with Formal Privacy Guarantees: State of the Art and the Road Ahead](https://mlsanity.com/paper/2503.20846/) (2025)
- [RAPID: Retrieval Augmented Training of Differentially Private Diffusion Models](https://mlsanity.com/paper/2502.12794/) (2025)
- [Efficient Differentially Private Fine-Tuning of Diffusion Models](https://mlsanity.com/paper/2406.05257/) (2024)
- [Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Training](https://mlsanity.com/paper/2410.19941/) (2024)
- [Differentially Private Kernel Inducing Points using features from ScatterNets (DP-KIP-ScatterNet) for Privacy Preserving Data Distillation](https://mlsanity.com/paper/2301.13389/) (2023)
- [PAC Privacy Preserving Diffusion Models](https://mlsanity.com/paper/2312.01201/) (2023)
- [Bayesian Power Steering: An Effective Approach for Domain Adaptation of Diffusion Models](https://mlsanity.com/paper/2406.03683/) (2024)
- [Optimal Differentially Private Sampling of Unbounded Gaussians](https://mlsanity.com/paper/2503.01766/) (2025)
- [DP-DocLDM: Differentially Private Document Image Generation using Latent Diffusion Models](https://mlsanity.com/paper/2508.04208/) (2025)
- [DPAF: Image Synthesis via Differentially Private Aggregation in Forward Phase](https://mlsanity.com/paper/2304.12185/) (2023)
- [DPGOMI: Differentially Private Data Publishing with Gaussian Optimized Model Inversion](https://mlsanity.com/paper/2310.04528/) (2023)
- [Private Training & Data Generation by Clustering Embeddings](https://mlsanity.com/paper/2506.16661/) (2025)
- [From Easy to Hard++: Promoting Differentially Private Image Synthesis Through Spatial-Frequency Curriculum](https://mlsanity.com/paper/2601.06368/) (2026)
- [Rethinking Benchmarks for Differentially Private Image Classification](https://mlsanity.com/paper/2601.17189/) (2026)
- [DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning](https://mlsanity.com/paper/2602.18633/) (2026)
- [Protecting and Preserving Protest Dynamics for Responsible Analysis](https://mlsanity.com/paper/2604.05256/) (2026)

## Abstract (arXiv)

Training even moderately-sized generative models with differentially-private stochastic gradient descent (DP-SGD) is difficult: the required level of noise for reasonable levels of privacy is simply too large. We advocate instead building off a good, relevant representation on an informative public dataset, then learning to model the private data with that representation. In particular, we minimize the maximum mean discrepancy (MMD) between private target data and a generator's distribution, using a kernel based on perceptual features learned from a public dataset. With the MMD, we can simply privatize the data-dependent term once and for all, rather than introducing noise at each step of optimization as in DP-SGD. Our algorithm allows us to generate CIFAR10-level images with ε ≈ 2 which capture distinctive features in the distribution, far surpassing the current state of the art, which mostly focuses on datasets such as MNIST and FashionMNIST at a large ε ≈ 10. Our work introduces simple yet powerful foundations for reducing the gap between private and non-private deep generative models. Our code is available at urlhttps://github.com/ParkLabML/DP-MEPF.