# Iterative Methods for Private Synthetic Data: Unifying Framework and New Methods

arXiv: https://arxiv.org/abs/2106.07153 · HTML: https://mlsanity.com/paper/2106.07153/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds](https://mlsanity.com/paper/1605.02065/) (2016)
- [From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification](https://mlsanity.com/paper/1602.02068/) (2016)
- [Differentially Private Generative Adversarial Network](https://mlsanity.com/paper/1802.06739/) (2018)
- [Concentrated Differential Privacy](https://mlsanity.com/paper/1603.01887/) (2016)
- [Fingerprinting Codes and the Price of Approximate Differential Privacy](https://mlsanity.com/paper/1311.3158/) (2013)
- [Graphical-model based estimation and inference for differential privacy](https://mlsanity.com/paper/1901.09136/) (2019)
- [The Unusual Effectiveness of Averaging in GAN Training](https://mlsanity.com/paper/1806.04498/) (2018)
- [Differentially Private Query Release Through Adaptive Projection](https://mlsanity.com/paper/2103.06641/) (2021)
- [New Oracle-Efficient Algorithms for Private Synthetic Data Release](https://mlsanity.com/paper/2007.05453/) (2020)
- [Limits of Private Learning with Access to Public Data](https://mlsanity.com/paper/1910.11519/) (2019)
- [Leveraging Public Data for Practical Private Query Release](https://mlsanity.com/paper/2102.08598/) (2021)
- [Decision Making with Differential Privacy under a Fairness Lens](https://mlsanity.com/paper/2105.07513/) (2021)
- [The Power of Factorization Mechanisms in Local and Central Differential Privacy](https://mlsanity.com/paper/1911.08339/) (2019)
- [Optimizing error of high-dimensional statistical queries under differential privacy](https://mlsanity.com/paper/1808.03537/) (2018)
- [Private Query Release Assisted by Public Data](https://mlsanity.com/paper/2004.10941/) (2020)
- [Dual Query: Practical Private Query Release for High Dimensional Data](https://mlsanity.com/paper/1402.1526/) (2014)
- [Private Post-GAN Boosting](https://mlsanity.com/paper/2007.11934/) (2020)
- [Learning from Mixtures of Private and Public Populations](https://mlsanity.com/paper/2008.00331/) (2020)
- [Private Multiplicative Weights Beyond Linear Queries](https://mlsanity.com/paper/1407.1571/) (2014)
- [Bounding, Concentrating, and Truncating: Unifying Privacy Loss Composition for Data Analytics](https://mlsanity.com/paper/2004.07223/) (2020)

## Built on by (most cited first, 30 of 59)

- [How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy](https://mlsanity.com/paper/2303.00654/) (2023)
- [Harnessing large-language models to generate private synthetic text](https://mlsanity.com/paper/2306.01684/) (2023)
- [Private Synthetic Data for Multitask Learning and Marginal Queries](https://mlsanity.com/paper/2209.07400/) (2022)
- [Privacy-Preserving Instructions for Aligning Large Language Models](https://mlsanity.com/paper/2402.13659/) (2024)
- [A Linear Reconstruction Approach for Attribute Inference Attacks against Synthetic Data](https://mlsanity.com/paper/2301.10053/) (2023)
- [Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)](https://mlsanity.com/paper/2407.12858/) (2024)
- [Generating Private Synthetic Data with Genetic Algorithms](https://mlsanity.com/paper/2306.03257/) (2023)
- [Confidence-Ranked Reconstruction of Census Microdata from Published Statistics](https://mlsanity.com/paper/2211.03128/) (2022)
- [Differentially Private Tabular Data Synthesis using Large Language Models](https://mlsanity.com/paper/2406.01457/) (2024)
- [A Survey on Deep Learning Approaches for Tabular Data Generation: Utility, Alignment, Fidelity, Privacy, Diversity, and Beyond](https://mlsanity.com/paper/2503.05954/) (2025)
- [Graphical vs. Deep Generative Models: Measuring the Impact of Differentially Private Mechanisms and Budgets on Utility](https://mlsanity.com/paper/2305.10994/) (2023)
- [CuTS: Customizable Tabular Synthetic Data Generation](https://mlsanity.com/paper/2307.03577/) (2023)
- [DP-TBART: A Transformer-based Autoregressive Model for Differentially Private Tabular Data Generation](https://mlsanity.com/paper/2307.10430/) (2023)
- [The Elusive Pursuit of Reproducing PATE-GAN: Benchmarking, Auditing, Debugging](https://mlsanity.com/paper/2406.13985/) (2024)
- [dpart: Differentially Private Autoregressive Tabular, a General Framework for Synthetic Data Generation](https://mlsanity.com/paper/2207.05810/) (2022)
- [The DCR Delusion: Measuring the Privacy Risk of Synthetic Data](https://mlsanity.com/paper/2505.01524/) (2025)
- [InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy](https://mlsanity.com/paper/2507.02974/) (2025)
- [Noise-Aware Statistical Inference with Differentially Private Synthetic Data](https://mlsanity.com/paper/2205.14485/) (2022)
- [Post-processing Private Synthetic Data for Improving Utility on Selected Measures](https://mlsanity.com/paper/2305.15538/) (2023)
- [Synthetic Data in Healthcare](https://mlsanity.com/paper/2304.03243/) (2023)
- [Privacy-preserving data release leveraging optimal transport and particle gradient descent](https://mlsanity.com/paper/2401.17823/) (2024)
- [Relaxed Marginal Consistency for Differentially Private Query Answering](https://mlsanity.com/paper/2109.06153/) (2021)
- [FLAIM: AIM-based Synthetic Data Generation in the Federated Setting](https://mlsanity.com/paper/2310.03447/) (2023)
- [Oracle-Efficient Differentially Private Learning with Public Data](https://mlsanity.com/paper/2402.09483/) (2024)
- [Tabular Data Synthesis with Differential Privacy: A Survey](https://mlsanity.com/paper/2411.03351/) (2024)
- [Privately generating tabular data using language models](https://mlsanity.com/paper/2306.04803/) (2023)
- [Is API Access to LLMs Useful for Generating Private Synthetic Tabular Data?](https://mlsanity.com/paper/2502.06555/) (2025)
- [The Importance of Being Discrete: Measuring the Impact of Discretization in End-to-End Differentially Private Synthetic Data](https://mlsanity.com/paper/2504.06923/) (2025)
- [A Unified View of Differentially Private Deep Generative Modeling](https://mlsanity.com/paper/2309.15696/) (2023)
- [A Simple and Practical Method for Reducing the Disparate Impact of Differential Privacy](https://mlsanity.com/paper/2312.11712/) (2023)

## Abstract (arXiv)

We study private synthetic data generation for query release, where the goal is to construct a sanitized version of a sensitive dataset, subject to differential privacy, that approximately preserves the answers to a large collection of statistical queries. We first present an algorithmic framework that unifies a long line of iterative algorithms in the literature. Under this framework, we propose two new methods. The first method, private entropy projection (PEP), can be viewed as an advanced variant of MWEM that adaptively reuses past query measurements to boost accuracy. Our second method, generative networks with the exponential mechanism (GEM), circumvents computational bottlenecks in algorithms such as MWEM and PEP by optimizing over generative models parameterized by neural networks, which capture a rich family of distributions while enabling fast gradient-based optimization. We demonstrate that PEP and GEM empirically outperform existing algorithms. Furthermore, we show that GEM nicely incorporates prior information from public data while overcoming limitations of PMW^Pub, the existing state-of-the-art method that also leverages public data.