# Hermite Polynomial Features for Private Data Generation

arXiv: https://arxiv.org/abs/2106.05042 · HTML: https://mlsanity.com/paper/2106.05042/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 42 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Differentially Private Generative Adversarial Network](https://mlsanity.com/paper/1802.06739/) (2018)
- [DP-CGAN: Differentially Private Synthetic Data and Label Generation](https://mlsanity.com/paper/2001.09700/) (2020)
- [GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators](https://mlsanity.com/paper/2006.08265/) (2020)
- [DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generation](https://mlsanity.com/paper/2002.11603/) (2020)
- [Graphical-model based estimation and inference for differential privacy](https://mlsanity.com/paper/1901.09136/) (2019)
- [Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels](https://mlsanity.com/paper/1412.8293/) (2014)
- [Differentially Private Mixture of Generative Neural Networks](https://mlsanity.com/paper/1709.04514/) (2017)
- [Differentially Private Generative Adversarial Networks for Time Series, Continuous, and Discrete Open Data](https://mlsanity.com/paper/1901.02477/) (2019)
- [Differentially Private Database Release via Kernel Mean Embeddings](https://mlsanity.com/paper/1710.01641/) (2017)
- [pMSE Mechanism: Differentially Private Synthetic Data with Maximal Distributional Similarity](https://mlsanity.com/paper/1805.09392/) (2018)
- [Ektelo: A Framework for Defining Differentially-Private Computations](https://mlsanity.com/paper/1808.03555/) (2018)
- [Differentially Private Distributed Data Summarization under Covariate Shift](https://mlsanity.com/paper/1910.12832/) (2019)

## Built on by (most cited first, 21 of 21)

- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [Differentially Private Diffusion Models](https://mlsanity.com/paper/2210.09929/) (2022)
- [Differentially Private Synthetic Data via Foundation Model APIs 1: Images](https://mlsanity.com/paper/2305.15560/) (2023)
- [SoK: Privacy-Preserving Data Synthesis](https://mlsanity.com/paper/2307.02106/) (2023)
- [DP-LDMs: Differentially Private Latent Diffusion Models](https://mlsanity.com/paper/2305.15759/) (2023)
- [Pre-trained Perceptual Features Improve Differentially Private Image Generation](https://mlsanity.com/paper/2205.12900/) (2022)
- [Private GANs, Revisited](https://mlsanity.com/paper/2302.02936/) (2023)
- [Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generation](https://mlsanity.com/paper/2303.01687/) (2023)
- [Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model](https://mlsanity.com/paper/2502.05505/) (2025)
- [InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy](https://mlsanity.com/paper/2507.02974/) (2025)
- [Tabular Data Synthesis with Differential Privacy: A Survey](https://mlsanity.com/paper/2411.03351/) (2024)
- [A Unified View of Differentially Private Deep Generative Modeling](https://mlsanity.com/paper/2309.15696/) (2023)
- [PFGuard: A Generative Framework with Privacy and Fairness Safeguards](https://mlsanity.com/paper/2410.02246/) (2024)
- [RAPID: Retrieval Augmented Training of Differentially Private Diffusion Models](https://mlsanity.com/paper/2502.12794/) (2025)
- [NeuroMixGDP: A Neural Collapse-Inspired Random Mixup for Private Data Release](https://mlsanity.com/paper/2202.06467/) (2022)
- [Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Training](https://mlsanity.com/paper/2410.19941/) (2024)
- [Private Text Generation by Seeding Large Language Model Prompts](https://mlsanity.com/paper/2502.13193/) (2025)
- [Differentially Private Kernel Inducing Points using features from ScatterNets (DP-KIP-ScatterNet) for Privacy Preserving Data Distillation](https://mlsanity.com/paper/2301.13389/) (2023)
- [DPAF: Image Synthesis via Differentially Private Aggregation in Forward Phase](https://mlsanity.com/paper/2304.12185/) (2023)
- [VP-NTK: Exploring the Benefits of Visual Prompting in Differentially Private Data Synthesis](https://mlsanity.com/paper/2503.16195/) (2025)
- [StraightDP: Geometry-Aware Differential Privacy for Rectified-Flow Transformers](https://mlsanity.com/paper/2607.29100/) (2026)

## Abstract (arXiv)

Kernel mean embedding is a useful tool to represent and compare probability measures. Despite its usefulness, kernel mean embedding considers infinite-dimensional features, which are challenging to handle in the context of differentially private data generation. A recent work proposes to approximate the kernel mean embedding of data distribution using finite-dimensional random features, which yields analytically tractable sensitivity. However, the number of required random features is excessively high, often ten thousand to a hundred thousand, which worsens the privacy-accuracy trade-off. To improve the trade-off, we propose to replace random features with Hermite polynomial features. Unlike the random features, the Hermite polynomial features are ordered, where the features at the low orders contain more information on the distribution than those at the high orders. Hence, a relatively low order of Hermite polynomial features can more accurately approximate the mean embedding of the data distribution compared to a significantly higher number of random features. As demonstrated on several tabular and image datasets, Hermite polynomial features seem better suited for private data generation than random Fourier features.