mlsanityData to 14 Oct 2026

Differentially Private Language Models for Secure Data Sharing

arXiv 2210.13918 ↗First posted 25 Oct 2022cs.LG cs.CL cs.CRcited by 68

Not on the map. Why?

Abstract (arXiv)

To protect the privacy of individuals whose data is being shared, it is of high importance to develop methods allowing researchers and companies to release textual data while providing formal privacy guarantees to its originators. In the field of NLP, substantial efforts have been directed at building mechanisms following the framework of local differential privacy, thereby anonymizing individual text samples before releasing them. In practice, these approaches are often dissatisfying in terms of the quality of their output language due to the strong noise required for local differential privacy. In this paper, we approach the problem at hand using global differential privacy, particularly by training a generative language model in a differentially private manner and consequently sampling data from it. Using natural language prompts and a new prompt-mismatch loss, we are able to create highly accurate and fluent textual datasets taking on specific desired attributes such as sentiment or topic and resembling statistical properties of the training data. We perform thorough experiments indicating that our synthetic datasets do not leak information from our original data and are of high language quality and highly suitable for training models for further analysis on real-world data. Notably, we also demonstrate that training classifiers on private synthetic data outperforms directly training classifiers on real data with DP-SGD.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Builds on 46 of its 76 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

46 of 76 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  8. 2016Deep Learning with Differential Privacycited by 2,728
  9. 2020Extracting Training Data from Large Language Modelscited by 2,189
  10. 2017Renyi Differential Privacycited by 650
  11. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  12. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
34 more
  1. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  2. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  3. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  4. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 346
  5. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  6. 2013The Composition Theorem for Differential Privacycited by 297
  7. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
  8. 2018Differentially Private Generative Adversarial Networkcited by 235
  9. 2020Information Leakage in Embedding Modelscited by 203
  10. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  11. 2021Generating Datasets with Pretrained Language Modelscited by 193
  12. 2021Numerical Composition of Differential Privacycited by 180
  13. 2019Not Enough Data? Deep Learning to the Rescue!cited by 175
  14. 2016Concentrated Differential Privacycited by 144
  15. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 119
  16. 2020DP-CGAN: Differentially Private Synthetic Data and Label Generationcited by 118
  17. 2021Large-Scale Differentially Private BERTcited by 118
  18. 2020GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generatorscited by 117
  19. 2019Deep Learning with Gaussian Differential Privacycited by 114
  20. 2020Few-Shot Text Generation with Pattern-Exploiting Trainingcited by 87
  21. 2022The Limits of Word Level Differential Privacycited by 63
  22. 2018Privacy-preserving Neural Representations of Textcited by 54
  23. 2018Generalised Differential Privacy for Text Document Processingcited by 52
  24. 2019Leveraging Hierarchical Representations for Preserving Privacy and Utility in Textcited by 50
  25. 2017A4NT: Author Attribute Anonymity by Adversarial Training of Neural Machine Translationcited by 42
  26. 2019ER-AE: Differentially Private Text Generation for Authorship Anonymizationcited by 35
  27. 2021StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfercited by 34
  28. 2021ADePT: Auto-encoder based Differentially Private Text Transformationcited by 31
  29. 2021When differential privacy meets NLP: The devil is in the detailcited by 25
  30. 2018SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Miningcited by 21
  31. 2020Private Post-GAN Boostingcited by 20
  32. 2019I Am Not What I Write: Privacy Preserving Text Representation Learningcited by 12
  33. 2022How reparametrization trick broke differentially-private text representation learningcited by 10
  34. 2018Blind De-anonymization Attacks using Social Networkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 12; 2024: 19; 2025: 24; 2026: 11.

68 papers.

Sort:

By area:

  1. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 96
  2. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Textcited by 76
  3. 2023Harnessing large-language models to generate private synthetic textcited by 65
  4. 2023Locally Differentially Private Document Generation Using Zero Shot Promptingcited by 63
  5. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
  6. 2024ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluationcited by 47
  7. 2024LLM-PBE: Assessing Data Privacy in Large Language Modelscited by 41
  8. 2023PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Modelscited by 37
  9. 2022MAUVE Scores for Generative Models: Theory and Practicecited by 36
  10. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 35
  11. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  12. 2023DP-BART for Privatized Text Rewriting under Local Differential Privacycited by 28
38 more
  1. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 28
  2. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  3. 2024Differentially Private Knowledge Distillation via Synthetic Text Generationcited by 18
  4. 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Textcited by 18
  5. 2023Privately Aligning Language Models with Reinforcement Learningcited by 17
  6. 2024Differentially Private Tabular Data Synthesis using Large Language Modelscited by 16
  7. 2024HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protectioncited by 16
  8. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 16
  9. 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 16
  10. 2024Differential Privacy Mechanisms in Neural Tangent Kernel Regressioncited by 14
  11. 2024Provable Differentially Private Computation of the Cross-Attention Mechanismcited by 14
  12. 2025POPri: Private Federated Learning using Preference-Optimized Synthetic Datacited by 14
  13. 2022Privacy-Preserving Domain Adaptation of Semantic Parserscited by 12
  14. 2024Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domainscited by 12
  15. 2023Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Modelscited by 10
  16. 2024Robust Utility-Preserving Text Anonymization Based on Large Language Modelscited by 10
  17. 2024Data-adaptive Differentially Private Prompt Synthesis for In-Context Learningcited by 10
  18. 2024Private Synthetic Text Generation with Diffusion Modelscited by 10
  19. 2025InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacycited by 10
  20. 2024LLM-based Privacy Data Augmentation Guided by Knowledge Distillation with a Distribution Tutor for Medical Text Classificationcited by 8
  21. 2025Synthesize Privacy-Preserving High-Resolution Images via Private Textual Intermediariescited by 8
  22. 2023Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Modelscited by 7
  23. 2023DP-NMT: Scalable Differentially-Private Machine Translationcited by 7
  24. 2025From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesiscited by 7
  25. 2025SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domainscited by 7
  26. 2024TAROT: Task-Oriented Authorship Obfuscation Using Policy Optimization Methodscited by 5
  27. 2025Clustering and Median Aggregation Improve Differentially Private Inferencecited by 5
  28. 2025Evaluating Differentially Private Generation of Domain-Specific Textcited by 5
  29. 2023A Unified View of Differentially Private Deep Generative Modelingcited by 4
  30. 2024Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Datacited by 4
  31. 2025Generating Synthetic Data with Formal Privacy Guarantees: State of the Art and the Road Aheadcited by 4
  32. 2025SynBench: A Benchmark for Differentially Private Text Generationcited by 4
  33. 2025RewardDS: Privacy-Preserving Fine-Tuning for Large Language Models via Reward Driven Data Synthesiscited by 3
  34. 2025ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Controlcited by 3
  35. 2026Privasis: Synthesizing the Largest "Public" Private Dataset from Scratchcited by 3
  36. 2026DP-OPD: Differentially Private On-Policy Distillation for Language Modelscited by 3
  37. 2024Federated Domain-Specific Knowledge Transfer on Large Language Models Using Synthetic Datacited by 2
  38. 2025Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outscited by 2

50 most cited of 68 All 68 (JSON)