mlsanityData to 14 Oct 2026

Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe

arXiv 2210.14348 ↗First posted 25 Oct 2022cs.CL cs.CRcited by 108

Not on the map. Why?

Abstract (arXiv)

Privacy concerns have attracted increasing attention in data-driven products due to the tendency of machine learning models to memorize sensitive training data. Generating synthetic versions of such data with a formal privacy guarantee, such as differential privacy (DP), provides a promising path to mitigating these privacy concerns, but previous approaches in this direction have typically failed to produce synthetic data of high quality. In this work, we show that a simple and practical recipe in the text domain is effective: simply fine-tuning a pretrained generative language model with DP enables the model to generate useful synthetic text with strong privacy protection. Through extensive empirical analyses on both benchmark and private customer data, we demonstrate that our method produces synthetic text that is competitive in terms of utility with its non-private counterpart, meanwhile providing strong protection against potential privacy leakages.

Key facts. Cited by 108 papers in the map (data through 14 Oct 2026). Builds on 49 of its 74 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

49 of 74 references.

References · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2016Deep Learning with Differential Privacycited by 2,728
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2020Extracting Training Data from Large Language Modelscited by 2,189
  9. 2018Hierarchical Neural Story Generationcited by 1,193
  10. 2019Machine Unlearningcited by 1,061
  11. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  12. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
37 more
  1. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  2. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  3. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  4. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  5. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  6. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  7. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  8. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 383
  9. 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 371
  10. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
  11. 2018Differentially Private Generative Adversarial Networkcited by 235
  12. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 224
  13. 2021Numerical Composition of Differential Privacycited by 180
  14. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 119
  15. 2021Winning the NIST Contest: A scalable and general approach to differentially private synthetic datacited by 102
  16. 2021Large Scale Private Learning via Low-rank Reparametrizationcited by 88
  17. 2021Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learningcited by 86
  18. 2019Generative Models for Effective ML on Private, Decentralized Datasetscited by 85
  19. 2021Benchmarking Differentially Private Synthetic Data Generation Algorithmscited by 81
  20. 2021Differential Privacy for Text Analytics via Natural Text Sanitizationcited by 77
  21. 2020Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorizationcited by 58
  22. 2021Natural Language Understanding with Privacy-Preserving BERTcited by 55
  23. 2019Leveraging Hierarchical Representations for Preserving Privacy and Utility in Textcited by 50
  24. 2022Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacycited by 48
  25. 2022When Does Differentially Private Learning Not Suffer in High Dimensions?cited by 48
  26. 2022Large Scale Transfer Learning for Differentially Private Image Classificationcited by 41
  27. 2020Chasing Your Long Tails: Differentially Private Prediction in Health Care Settingscited by 37
  28. 2014Differential Privacy: An Economic Method for Choosing Epsiloncited by 35
  29. 2019ER-AE: Differentially Private Text Generation for Authorship Anonymizationcited by 35
  30. 2021ADePT: Auto-encoder based Differentially Private Text Transformationcited by 31
  31. 2021When differential privacy meets NLP: The devil is in the detailcited by 25
  32. 2018SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Miningcited by 21
  33. 2020Private Post-GAN Boostingcited by 20
  34. 2021On a Utilitarian Approach to Privacy Preserving Text Generationcited by 17
  35. 2022Sentence-level Privacy for Document Embeddingscited by 16
  36. 2022Provably Confidential Language Modellingcited by 13
  37. 2021DTGAN: Differential Private Training for Tabular GANscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 16; 2024: 28; 2025: 37; 2026: 25.

108 papers.

Sort:

By area:

  1. 2023Machine Learning for Synthetic Data Generation: A Reviewcited by 120
  2. 2024Machine Unlearning of Pre-trained Large Language Modelscited by 119
  3. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 96
  4. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Textcited by 76
  5. 2023Differentially Private Synthetic Data via Foundation Model APIs 1: Imagescited by 65
  6. 2023Harnessing large-language models to generate private synthetic textcited by 65
  7. 2023Right to be Forgotten in the Era of Large Language Models: Implications, Challenges, and Solutionscited by 60
  8. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
  9. 2024LLM-PBE: Assessing Data Privacy in Large Language Modelscited by 41
  10. 2023PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Modelscited by 37
  11. 2022MAUVE Scores for Generative Models: Theory and Practicecited by 36
  12. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 35
38 more
  1. 2023How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilitiescited by 33
  2. 2023Privacy Implications of Retrieval-Based Language Modelscited by 32
  3. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  4. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 28
  5. 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agentscited by 26
  6. 2024Privacy Preserving Prompt Engineering: A Surveycited by 25
  7. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  8. 2024Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuningcited by 24
  9. 2024Private prediction for large-scale synthetic text generationcited by 24
  10. 2025SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Predictioncited by 23
  11. 2024Synthetic Data in AI: Challenges, Applications, and Ethical Implicationscited by 21
  12. 2023Differentially Private Natural Language Models: Recent Advances and Future Directionscited by 20
  13. 2025The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaborationcited by 20
  14. 2024Differentially Private Knowledge Distillation via Synthetic Text Generationcited by 18
  15. 2024Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challengescited by 18
  16. 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Textcited by 18
  17. 2023Privately Aligning Language Models with Reinforcement Learningcited by 17
  18. 2024Prompt Public Large Language Models to Synthesize Data for Private On-device Applicationscited by 17
  19. 2024Differentially Private Tabular Data Synthesis using Large Language Modelscited by 16
  20. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 16
  21. 2025Cultural Learning-Based Culture Adaptation of Language Modelscited by 16
  22. 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 16
  23. 2024De-identification is not enough: a comparison between de-identified and synthetic clinical notescited by 14
  24. 2025POPri: Private Federated Learning using Preference-Optimized Synthetic Datacited by 14
  25. 2024The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Promptscited by 13
  26. 2025Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Modelcited by 13
  27. 2022Privacy-Preserving Domain Adaptation of Semantic Parserscited by 12
  28. 2023Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Surveycited by 12
  29. 2024Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domainscited by 12
  30. 2024KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Servercited by 11
  31. 2025DPImageBench: A Unified Benchmark for Differentially Private Image Synthesiscited by 11
  32. 2023Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Modelscited by 10
  33. 20241-Diffractor: Efficient and Utility-Preserving Text Obfuscation Leveraging Word-Level Metric Differential Privacycited by 10
  34. 2024Data-adaptive Differentially Private Prompt Synthesis for In-Context Learningcited by 10
  35. 2024Private Synthetic Text Generation with Diffusion Modelscited by 10
  36. 2025InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacycited by 10
  37. 2024Efficiently Computing Similarities to Private Datasetscited by 9
  38. 2024LLM-based Privacy Data Augmentation Guided by Knowledge Distillation with a Distribution Tutor for Medical Text Classificationcited by 8

50 most cited of 108 All 108 (JSON)