mlsanityData to 7 Oct 2026

Iterative Methods for Private Synthetic Data: Unifying Framework and New Methods

arXiv 2106.07153 ↗First posted 14 Jun 2021cs.LG cs.CR cs.DScited by 58

Not on the map. Why?

Abstract (arXiv)

We study private synthetic data generation for query release, where the goal is to construct a sanitized version of a sensitive dataset, subject to differential privacy, that approximately preserves the answers to a large collection of statistical queries. We first present an algorithmic framework that unifies a long line of iterative algorithms in the literature. Under this framework, we propose two new methods. The first method, private entropy projection (PEP), can be viewed as an advanced variant of MWEM that adaptively reuses past query measurements to boost accuracy. Our second method, generative networks with the exponential mechanism (GEM), circumvents computational bottlenecks in algorithms such as MWEM and PEP by optimizing over generative models parameterized by neural networks, which capture a rich family of distributions while enabling fast gradient-based optimization. We demonstrate that PEP and GEM empirically outperform existing algorithms. Furthermore, we show that GEM nicely incorporates prior information from public data while overcoming limitations of PMW^Pub, the existing state-of-the-art method that also leverages public data.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Builds on 21 of its 45 references that are in the map. First posted on arXiv 14 Jun 2021.

Builds on

21 of 45 references.

References · 21

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  3. 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 337
  4. 2018Differentially Private Generative Adversarial Networkcited by 235
  5. 2016Concentrated Differential Privacycited by 139
  6. 2013Fingerprinting Codes and the Price of Approximate Differential Privacycited by 81
  7. 2018The Unusual Effectiveness of Averaging in GAN Trainingcited by 73
  8. 2019Graphical-model based estimation and inference for differential privacycited by 73
  9. 2021Differentially Private Query Release Through Adaptive Projectioncited by 50
  10. 2020New Oracle-Efficient Algorithms for Private Synthetic Data Releasecited by 43
  11. 2021Leveraging Public Data for Practical Private Query Releasecited by 39
  12. 2019Limits of Private Learning with Access to Public Datacited by 38
9 more
  1. 2021Decision Making with Differential Privacy under a Fairness Lenscited by 32
  2. 2019The Power of Factorization Mechanisms in Local and Central Differential Privacycited by 30
  3. 2018Optimizing error of high-dimensional statistical queries under differential privacycited by 29
  4. 2020Private Query Release Assisted by Public Datacited by 27
  5. 2014Dual Query: Practical Private Query Release for High Dimensional Datacited by 26
  6. 2020Private Post-GAN Boostingcited by 20
  7. 2020Learning from Mixtures of Private and Public Populationscited by 19
  8. 2014Private Multiplicative Weights Beyond Linear Queriescited by 17
  9. 2020Bounding, Concentrating, and Truncating: Unifying Privacy Loss Composition for Data Analyticscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 7; 2023: 15; 2024: 15; 2025: 14; 2026: 6.

58 papers.

Sort:

By area:

  1. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  2. 2023Harnessing large-language models to generate private synthetic textcited by 65
  3. 2022Private Synthetic Data for Multitask Learning and Marginal Queriescited by 35
  4. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 34
  5. 2023A Linear Reconstruction Approach for Attribute Inference Attacks against Synthetic Datacited by 23
  6. 2024Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)cited by 20
  7. 2023Generating Private Synthetic Data with Genetic Algorithmscited by 18
  8. 2024Differentially Private Tabular Data Synthesis using Large Language Modelscited by 16
  9. 2022Confidence-Ranked Reconstruction of Census Microdata from Published Statisticscited by 15
  10. 2023Graphical vs. Deep Generative Models: Measuring the Impact of Differentially Private Mechanisms and Budgets on Utilitycited by 13
  11. 2023DP-TBART: A Transformer-based Autoregressive Model for Differentially Private Tabular Data Generationcited by 13
  12. 2025A Survey on Deep Learning Approaches for Tabular Data Generation: Utility, Alignment, Fidelity, Privacy, Diversity, and Beyondcited by 13
38 more
  1. 2022dpart: Differentially Private Autoregressive Tabular, a General Framework for Synthetic Data Generationcited by 12
  2. 2023CuTS: Customizable Tabular Synthetic Data Generationcited by 12
  3. 2024The Elusive Pursuit of Reproducing PATE-GAN: Benchmarking, Auditing, Debuggingcited by 12
  4. 2025The DCR Delusion: Measuring the Privacy Risk of Synthetic Datacited by 11
  5. 2025InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacycited by 10
  6. 2022Noise-Aware Statistical Inference with Differentially Private Synthetic Datacited by 9
  7. 2023Post-processing Private Synthetic Data for Improving Utility on Selected Measurescited by 9
  8. 2023Synthetic Data in Healthcarecited by 8
  9. 2024Privacy-preserving data release leveraging optimal transport and particle gradient descentcited by 8
  10. 2023FLAIM: AIM-based Synthetic Data Generation in the Federated Settingcited by 6
  11. 2024Tabular Data Synthesis with Differential Privacy: A Surveycited by 6
  12. 2021Relaxed Marginal Consistency for Differentially Private Query Answeringcited by 5
  13. 2023Privately generating tabular data using language modelscited by 5
  14. 2024Oracle-Efficient Differentially Private Learning with Public Datacited by 5
  15. 2025Is API Access to LLMs Useful for Generating Private Synthetic Tabular Data?cited by 5
  16. 2023A Unified View of Differentially Private Deep Generative Modelingcited by 4
  17. 2023A Simple and Practical Method for Reducing the Disparate Impact of Differential Privacycited by 4
  18. 2024Private Regression via Data-Dependent Sufficient Statistic Perturbationcited by 4
  19. 2025The Importance of Being Discrete: Measuring the Impact of Discretization in End-to-End Differentially Private Synthetic Datacited by 4
  20. 2025Improving the Generation and Evaluation of Synthetic Data for Downstream Medical Causal Inferencecited by 4
  21. 2024Bounding the Excess Risk for Linear Models Trained on Marginal-Preserving, Differentially-Private, Synthetic Datacited by 3
  22. 2024Sharper Bounds for Chebyshev Moment Matching, with Applicationscited by 3
  23. 2022Private Synthetic Data with Hierarchical Structurecited by 2
  24. 2022MIME: Minority Inclusion for Majority Group Enhancement of AI Performancecited by 2
  25. 2022Answering Private Linear Queries Adaptively using the Common Mechanismcited by 2
  26. 2023ResidualPlanner+: a scalable matrix mechanism for marginals and beyondcited by 2
  27. 2024A Bias-Variance Decomposition for Ensembles over Multiple Synthetic Datasetscited by 2
  28. 2024Differentially Private Synthetic Data Generation for Relational Databasescited by 2
  29. 2024Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Trainingcited by 2
  30. 2025RAPID: Retrieval Augmented Training of Differentially Private Diffusion Modelscited by 2
  31. 2025Leveraging Vertical Public-Private Split for Improved Synthetic Data Generationcited by 2
  32. 2025dpmm: Differentially Private Marginal Models, a Library for Synthetic Tabular Data Generationcited by 2
  33. 2026Differentially Private Synthetic Data via APIs 4: Tabular Datacited by 2
  34. 2023Private Synthetic Data Meets Ensemble Learningcited by 1
  35. 2024Efficient and Private Marginal Reconstruction with Local Non-Negativitycited by 1
  36. 2025PrAda-GAN: A Private Adaptive Generative Adversarial Network with Bayes Network Structurecited by 1
  37. 2025Operator-Theoretic Framework for Gradient-Free Federated Learningcited by 1
  38. 2026Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspectivecited by 1

50 most cited of 58 All 58 (JSON)