# DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generation

arXiv: https://arxiv.org/abs/2002.11603 · HTML: https://mlsanity.com/paper/2002.11603/

## Key facts

- Cited by 78 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2020.

## Built on by (most cited first, 30 of 78)

- [Privacy for Free: How does Dataset Condensation Help Privacy?](https://mlsanity.com/paper/2206.00240/) (2022)
- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [Differentially Private Diffusion Models](https://mlsanity.com/paper/2210.09929/) (2022)
- [Data Distillation: A Survey](https://mlsanity.com/paper/2301.04272/) (2023)
- [Differentially Private Diffusion Models Generate Useful Synthetic Images](https://mlsanity.com/paper/2302.13861/) (2023)
- [Differentially Private Synthetic Data via Foundation Model APIs 1: Images](https://mlsanity.com/paper/2305.15560/) (2023)
- [SoK: Privacy-Preserving Data Synthesis](https://mlsanity.com/paper/2307.02106/) (2023)
- [Private Synthetic Data for Multitask Learning and Marginal Queries](https://mlsanity.com/paper/2209.07400/) (2022)
- [Privacy-Preserving Instructions for Aligning Large Language Models](https://mlsanity.com/paper/2402.13659/) (2024)
- [Private Set Generation with Discriminative Information](https://mlsanity.com/paper/2211.04446/) (2022)
- [DP-LDMs: Differentially Private Latent Diffusion Models](https://mlsanity.com/paper/2305.15759/) (2023)
- [Pre-trained Perceptual Features Improve Differentially Private Image Generation](https://mlsanity.com/paper/2205.12900/) (2022)
- [PEARL: Data Synthesis via Private Embeddings and Adversarial Reconstruction Learning](https://mlsanity.com/paper/2106.04590/) (2021)
- [Hermite Polynomial Features for Private Data Generation](https://mlsanity.com/paper/2106.05042/) (2021)
- [Differentially Private Fine-Tuning of Diffusion Models](https://mlsanity.com/paper/2406.01355/) (2024)
- [Generating Private Synthetic Data with Genetic Algorithms](https://mlsanity.com/paper/2306.03257/) (2023)
- [Private GANs, Revisited](https://mlsanity.com/paper/2302.02936/) (2023)
- [Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generation](https://mlsanity.com/paper/2303.01687/) (2023)
- [A Survey on Deep Learning Approaches for Tabular Data Generation: Utility, Alignment, Fidelity, Privacy, Diversity, and Beyond](https://mlsanity.com/paper/2503.05954/) (2025)
- [DPD-fVAE: Synthetic Data Generation Using Federated Variational Autoencoders With Differentially-Private Decoder](https://mlsanity.com/paper/2211.11591/) (2022)
- [DP2-VAE: Differentially Private Pre-trained Variational Autoencoders](https://mlsanity.com/paper/2208.03409/) (2022)
- [DP-TBART: A Transformer-based Autoregressive Model for Differentially Private Tabular Data Generation](https://mlsanity.com/paper/2307.10430/) (2023)
- [PrivImage: Differentially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretraining](https://mlsanity.com/paper/2311.12850/) (2023)
- [Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model](https://mlsanity.com/paper/2502.05505/) (2025)
- [DPImageBench: A Unified Benchmark for Differentially Private Image Synthesis](https://mlsanity.com/paper/2503.14681/) (2025)
- [Noise-Aware Statistical Inference with Differentially Private Synthetic Data](https://mlsanity.com/paper/2205.14485/) (2022)
- [Synthetic Tabular Data: Methods, Attacks and Defenses](https://mlsanity.com/paper/2506.06108/) (2025)
- [On the Utility Recovery Incapability of Neural Net-based Differential Private Tabular Training Data Synthesizer under Privacy Deregulation](https://mlsanity.com/paper/2211.15809/) (2022)
- [From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesis](https://mlsanity.com/paper/2504.01395/) (2025)
- [Tabular Data Synthesis with Differential Privacy: A Survey](https://mlsanity.com/paper/2411.03351/) (2024)

## Abstract (arXiv)

We propose a differentially private data generation paradigm using random feature representations of kernel mean embeddings when comparing the distribution of true data with that of synthetic data. We exploit the random feature representations for two important benefits. First, we require a minimal privacy cost for training deep generative models. This is because unlike kernel-based distance metrics that require computing the kernel matrix on all pairs of true and synthetic data points, we can detach the data-dependent term from the term solely dependent on synthetic data. Hence, we need to perturb the data-dependent term only once and then use it repeatedly during the generator training. Second, we can obtain an analytic sensitivity of the kernel mean embedding as the random features are norm bounded by construction. This removes the necessity of hyper-parameter search for a clipping norm to handle the unknown sensitivity of a generator network. We provide several variants of our algorithm, differentially-private mean embeddings with random features (DP-MERF) to jointly generate labels and input features for datasets such as heterogeneous tabular data and image data. Our algorithm achieves drastically better privacy-utility trade-offs than existing methods when tested on several datasets.